An unintended consequence of the survey ‘redesign?’New research from the Employee benefit Research Institute (EBRI). Visit www.ebri.org
WASHINGTON –In recent years, the annual Social and Economic Supplement (the supplement) to the Current Population Survey (CPS) has shown significant declines in employment-based retirement plan participation. New research by EBRI, however, finds that this is likely misleading.
The supplement is one of the most-cited sources of income data for those whose ages are associated with being retired (typically ages 65 or older). In 2014, the U.S. Census Bureau conducted a redesign of the CPS questionnaire in an effort to improve income estimates being reported from the survey. However, after the CPS redesign – despite no changes to the retirement plan questions – the estimates of the percentage of workers who participated in an employment-based retirement plan decreased dramatically: from 64 percent under the old questionnaire to 58 percent under the redesigned questionnaire for the same year.
The participation numbers have continued to decline and now stand at 47 percent. “In analyzing the data, there appears to be a continuing unintended consequence of the survey redesign,” says Craig Copeland, of EBRI. Copeland points out that not only is there a discrepancy between the pre- and post-redesign numbers, the declines shown by the CPS data do not match other government sources. “When looking at the Bureau of Labor Statistics National Compensation Survey, for example, they do not reflect the decline in participation that is shown in the CPS.”
There is evidence that the redesign at least partially achieved its primary objective – increasing the level of income being reported under the CPS to better match other sources, such as IRS data. Still, Copeland notes, when comparing CPS data to IRS administrative date, it is clear that the CPS redesign is still not able to capture all of the income being reported to the IRS. As such, it is important to be careful about coming to conclusions about the retirement system based just on CPS data.
Excerpts from the Issue Brief ‘Current Population Survey’
The Annual Social and Economic Supplement (fielded in March of each year) to the Current Population Survey (CPS), conducted by the U.S. Census Bureau, is one of the most-cited sources of income data for those whose ages are associated with being retired (typically ages 65 or older).
In response to research showing that the survey misclassified and generally under-reported income, particularly pension income, the U.S. Census Bureau in 2014 conducted a redesign of the CPS questionnaire that altered and added to the questions on income. When such a change is made in a survey questionnaire, results can be affected–not just in the responses to changed questions, but also to results from other areas of the survey. From the CPS redesign, one such impact appeared to be on the estimates of the percentage of workers who participate in an employment-based retirement plan. Using this annual March CPS data, the Employee Benefit Research Institute (EBRI) has done annual reports on employment-based pension participation among civilian noninstitutionalized workers, including studies after the full
implementation of the survey redesign.
The basis of these studies are the two questions in the March CPS that ask workers if other than Social Security did any employer or union that you worked for in the prior year have a pension or other type of retirement plan for any of its employees and then whether the worker was included in that plan. These two questions were not part of the changes in the redesign of the income questions in the questionnaire.
Trends in Retirement Plan Participation
In the 2014 CPS, which provided results for 2013, a split-sample was used where both the traditional questionnaire (the questionnaire before any changes were made to the income questions) and the redesigned questionnaire (with the revised income questions) were used to conduct the survey.
As such, three sets of nationally represented results were available from the CPS that year: the traditional survey results, the redesigned survey results, and combined survey results where both the responses from the traditional and redesigned surveys were combined into one set of results.8 With the changes in the survey, the estimates of employment-based retirement plan participation were significantly different before and after the redesign (in 2013 and in 2011‒2012 compared with 2014‒2016), and the trends in participation have had substantially larger year-to-year changes after the redesign than before it.
Under the traditional survey design in 2014, the percentage of all civilian noninstitutionalized workers9 found to be participating in an employment-based retirement plan in 2013 was 40.8 percent, compared with 36.5 percent under the redesigned questionnaire, a difference of 4.3 percentage points. Looking at other workforce cohorts, for
Comparison with Bureau of Labor Statistics Data
Not only were the 2014–2016 declines found in the CPS estimates larger than at any point since at least 1987, the declines were not consistent with the findings from the Bureau of Labor Statistics’ National Compensation Survey (NCS). This survey found that the percentage of private-sector wage and salary workers at establishments with 500 or more employees participating in an employment-based retirement plan increased from 76 percent in 2013 to 77 percent in 2014, and then returned to 76 percent in 2015 and 2016.
In contrast, using the most comparable data in the CPS, the percentage of full-time, full-year wage and salary workers who worked for a private-sector employer with 500 or more employees was found to decrease from 64 percent for the traditional questionnaire and 58 percent from the redesigned questionnaire in 2013 to 55 percent in 2014, to 50 percent in 2015, and to 47 percent in 2016.
Comparison with Internal Revenue Service (IRS) Data
In a paper by Brady and Bass (2018), the authors present tabulations from the IRS’ Statistics of Income (SOI) Division that show that the percentage of workers with an employment-based retirement plan held steady from 2013 to 2014. This was in contrast to the result found in the CPS showing a decline during that period, even when just looking at the redesigned results. Furthermore, they show that the results from the SOI Form W-2 study indicate that the percentage of employees participating in an employment-based retirement plan has been higher than what has been found in the CPS for many years prior to the CPS questionnaire redesign.
The full Issue Brief is available here.
The Employee Benefit Research Institute is a private, nonpartisan, nonprofit research institute based in Washington, DC, that focuses on health, savings, retirement, and financial security issues. EBRI does not lobby and does not take policy positions. The work of EBRI is made possible by funding from its members and sponsors, which include a broad range of public, private, for-profit and nonprofit organizations. For more information go to www.ebri.org or visit the web site of EBRI’s affiliated American Savings Education Council at www.asec.org