Employment and unemployment numbers have a prominent position in the financial news, but many investors and commentators take an over simplified view of what they mean. All too often, a discussion of unemployment announcements is dismissed with the statement: "... is a lagging indicator.”
In two separate articles, I have discussed some ways that employment data can be used to determine where we are in the economic cycle and how to use the data to guide investment strategy. Many have criticized the published data as being unreliable because of modeling corrections and adjustments. In this article we will try to ask hard questions about the reliability of the data published.
What Are the Questions?
There has been a lot of chatter over the past few months about how to interpret employment reports and whether the unemployment numbers reported weekly and monthly by the U.S. Dept. of Labor are accurate, or even misleading. These are some of the questions:
- How many different surveys and reports are there?
- Are the weekly initial unemployment claims missing some people entering unemployment?
- Are the seasonal adjustments valid? Should the historical seasonal adjustment be applied when conditions deviate significantly from historical norms?
- How can it be that the increase in unemployment reported each month is a small fraction of the total of all weekly initial unemployment claims during that month?
- Should all people entering the workforce be added to the total of the employed (removed from the rolls of the unemployed) in the same way? Should temp employment be counted the same way as a permanent hire?
- Is the total number of employed people understating the number of people actually working? Are self-employed, part-time employed and people “off the books” counted?
A good summary of measurement and estimation methods and issues for monthly employment reports can be found here and here. Specific topics will be discussed in this article.
Why Are There Two Monthly Surveys?
The two surveys, the household survey and the establishment survey (also called the payroll survey), have two different objectives. The household survey objectives include estimating the employment status of the entire population and making some demographic assessments. The establishment survey objectives include assessing the industry and government group distributions of employment and making connections to unemployment claims data. If someone is working two jobs that are in the industry and government groups covered, that person could be counted twice if both jobs are surveyed. Multiple jobs are counted as one employed person in the household survey. More information about the two surveys can be obtained here and here.
In addition to the two monthly surveys, initial unemployment claims and continuing insured unemployed are reported weekly. In this article, both the monthly household employment survey and the weekly initial unemployment claims reports will be analyzed.
What Are Weekly Initial Unemployment Claims Missing?
Weekly initial unemployment claims do not count all people losing employment. Only those eligible for unemployment insurance benefits are counted. While some states cover all part time employment above some minimum number of hours, many states cover only full time employment. Part timers (in many but not all states), some temporary workers, contract workers, self-employed and off-the-books workers do not get picked up in this number when they lose their employment.
I do not have an estimate of how many people fall into these uncounted categories. It would not be surprising if it was several 10’s of thousands. There is no reason to suspect that these categories would not suffer loss of employment at a rate at least as high as covered employees. There are definitely significant numbers of people losing employment each week who are not counted. The official Insured Unemployed Rate as of May 2 is 4.8% (seasonally adjusted), which is a little more than half of the official total unemployment rate for April of 8.9%. The implication is that there may be a number of uninsured job losses each week, up to almost as many as the insured number.
Of course, there are people leaving the insured unemployed ranks each week, some who find jobs and some whose benefits expired. In the week of April 25, that number was 478,000. Since, as we shall see later in the article, there was an increase in the ranks of the employed of 120,000 for the entire month of April (the ranks of the unemployed increased by much more), the number of 478,000 is mostly composed of still unemployed people whose benefits ran out, probably more than 400,000.
What About Seasonal Adjustments?
Over the course of a year, the size of the nation's labor force and the levels of employment and unemployment undergo sharp fluctuations due to such seasonal events as changes in weather, reduced or expanded production, harvests, major holidays, and the opening and closing of schools.
Because these seasonal events follow a more or less regular pattern each year, their influence on statistical trends can be eliminated by adjusting the statistics from month to month.
While adjusting for past seasonal variables may be quite reproducible from year to year in an economy operating on an even keel, they may (or may not) be applicable in the same way in times of economic upheaval. This is an area that must be viewed as introducing possible bias when conditions in a certain month of the year are subjected to a much different economic environment than the same month in prior years. This uncertainty applies to both weekly initial unemployment claims and total unemployment estimates.
Monthly Employment Gains (Losses)
Let’s look at the recent report for April from the Dept. of Labor, released May 8.
One thing that stands out is the demographic distribution of unemployment. Teens and minorities are disproportionally unemployed. Another thing is the unemployment rate for men is 32% higher than women (9.4% vs. 7.1%). Only education, health services and government showed an increase in employment in April.
The biggest question about these numbers is the disparity between the increase in unemployment each month and the total of weekly initial unemployment claims. The 563,000 reported increase in unemployment for April is much smaller than 2,653,000 the total of weekly initial unemployment claims for the four April reporting dates.
Note: 2.6 million covers claims from March 29 through April 25, the four reporting weeks most closely aligned with the month of April.
The more recent data is given in the latest press release Thursday (May 28)
The numbers that should be reconciled in calculating the change in employment from one month to the next are shown in the following table, with Dept. of Labor published initial unemployment claims data and estimates (household survey data), where available.
The total of X – Y = 2,533,000 net new jobs created. In other words, 2.65 million put in initial unemployment claims, plus another Y uninsured lost their jobs in April. This was offset by 2.53 million X who found employment. The Dept. of Labor does not track these millions of individual moves. They simply estimate the number of employed (120,000 increase in April) and the number of unemployed (563,000 increase in April) for the month. These totals for the month are the results of millions of changes occurring in and out of employment.
Estimated Measurement Errors and Data Uncertainty
The Dept. of Labor makes the following statement regarding uncertainty in the reported numbers:
The establishment survey employment series has a smaller margin of error on the measurement of month-to-month change than the household survey because of its much larger sample size. An over-the-month employment change of 107,000 is statistically significant in the establishment survey, while the threshold for a statistically significant change in the household survey is about 400,000. However, the household survey has a more expansive scope than the establishment survey because it includes the self-employed, unpaid family workers, agricultural workers, and private household workers, who are excluded by the establishment survey.
The uncertainty referred to above is at the 90% confidence interval. This varies considerably from month to month. For example, in another section of Dept. of Labor literature, the following statement appears:
When a sample rather than the entire population is surveyed, there is a chance that the sample estimates may differ from the "true" population values they represent. The exact difference, or sampling error, varies depending on the particular sample selected, and this variability is measured by the standard error of the estimate. There is about a 90-percent chance, or level of confidence, that an estimate based on a sample will differ by no more than 1.6 standard errors from the "true" population value because of sampling error. BLS analyses are generally conducted at the 90-percent level of confidence. For example, the confidence interval for the monthly change in total employment from the household survey is on the order of plus or minus 430,000.
In the May 8 report covering April, the uncertainty is given as +/- 436,000 (table on page 4).
The household survey covers approximately 60,000 households each month. The following two table estimates on how this relates to the 18-65 age population.
We can use the numbers above to estimate how each response error reflects on the entire population age 18-65. The each individual covered by the survey represents approximately 2,400 people (235,000,000 divided by 97,800). Thus, an uncertainty of +/- 400,000 corresponds to approximately 167 employment status classification errors in each survey. This has to cover:
- Misunderstood questions
- Incorrect classification of responses
- Deliberately incorrect responses
- Processing errors after the interviews
If there are 50 survey interviewers (an arbitrarily assumed number - the Bureau of Labor Statistics says that 2000 people are involved in carrying out the household survey), that would allow about 24 minutes per survey for a 24 workday month (8 hours a day). This means that only 3.33 errors would occur per month for each interviewer. To verify this low number of errors, a study could be run to exactly duplicate surveys for several months, using different polling interviewers of the same respondents (and, in parallel, different respondents) and analyzing the results at each stage of the process. Has this ever been done?
It is certainly reasonable to question whether or not the sampling error is possibly understated. The larger the sampling error is each month, the more months must be accumulated in order to average out sampling errors and establish a trend. Perhaps we should be paying attention to the 2-, 3- or 4-month average of employment and unemployment figures.
The Birth/Death Adjustment
The Dept. of Labor makes a monthly model based estimate of jobs created by the birth of new businesses and jobs lost through the death of businesses.
The monthly establishment survey estimates include an adjustment to account for the net employment change generated by business births and deaths. The adjustment comes from an econometric model that forecasts the monthly net jobs impact of business births and deaths based on the actual past values of the net impact that can be observed with a lag from the Quarterly Census of Employment and Wages. The establishment survey uses modeling rather than sampling for this purpose because the survey is not immediately able to bring new businesses into the sample. There is an unavoidable lag between the birth of a new firm and its appearance on the sampling frame and availability for selection. BLS adds new businesses to the survey twice a year.
In April, the Dept. of Labor Bureau of Labor Statistics estimated that 226,000 jobs were created due the birth of new businesses. This number is net of losses due to business deaths. If this would change the numbers discussed previously (Monthly Employment Gains section), it would constitute a small fraction of the implied total of people finding new jobs. However, the previous discussion was based on the household survey, which it seems should include the effects of business births and deaths without adjustment, because the survey is not reaching people through businesses, it is surveying people directly (household survey). I have found the following questions and answers:
Q: Can I subtract the birth/death adjustment from the seasonally adjusted over-the-month change to determine what it is adding to employment?
A: No. Birth/death factors are a component of the not seasonally adjusted estimate and therefore are not directly comparable to the seasonally adjusted monthly changes. Instead, the birth/death factor should be assessed in the context of its effect on the not seasonally adjusted estimate.
Q: Can BLS provide an estimate of the contribution of the birth/death adjustment to the seasonally adjusted monthly payroll change?
A: BLS does not calculate an estimate of the seasonally adjusted contribution of the birth/death model. The sample, the imputation of business births using deaths, and the net birth/death model are all necessary components for obtaining an accurate total employment estimate. The components are not seasonally adjusted separately because they do not have any particular economic meaning in and of themselves.
This does not make clear whether the birth/death adjustment applies to the total employment numbers (from the household survey) or not. Reviewing three other publications, here, here and here, has not provided any further clarification.
Are People Falling Through The Cracks?
The Dept. of Labor maintains that their sampling techniques are getting good estimates for much of the labor force. The following table was constructed from April, 2009 seasonally adjusted data here and here.
This estimate of the total work age population seems a little high. Using the latest age distribution data from the U.S. Census Bureau (here) from July, 2007 and multiplying by 1.01 to correct for population growth since, I calculate the U.S. population from age 15 through 79 to be about 231,000,000.
It doesn’t look like the Dept. of Labor is missing people. If anything, they may have over counted slightly.
Even if the total working age population is about right, what about the 80 million in the age group but not in the work force. This presumably includes all not working because of long term illness and disability, and those not working by choice. The Dept. of Labor makes the following definitions:
The civilian labor force is the sum of employed and unemployed persons.
Those not classified as employed or unemployed are not in the labor force.
The unemployment rate is the number unemployed as a percent of the labor force.
The labor force participation rate is the labor force as a percent of the population, and the employment-population ratio is the employed as a percent of the population.
People are classified as employed if they did any work at all as paid employees during the reference week; worked in their own business, profession, or on their own farm; or worked without pay at least 15 hours in a family business or farm. People are also counted as employed if they were temporarily absent from their jobs because of illness, bad weather, vacation, labor-management disputes, or personal reasons.
People are classified as unemployed if they meet all of the following criteria:
- They had no employment during the reference week; and
- They were available for work at that time; and
- They made specific efforts to find employment sometime during the 4-week period ending with the reference week.
- Persons laid off from a job and expecting recall need not be looking for work to be counted as unemployed.
Note: The preceding statement separated into bullets by the author for readability.
So the people “not in the labor force” includes all people without employment (except for those with an anticipated future return from a layoff) who have not made a specific effort to find employment within the past four weeks. This is presumably a large number of people, probably larger during times of economic contraction. This group includes “marginally attached workers”, which includes “discouraged workers”. The April report stated:
About 2.1 million persons (not seasonally adjusted) were marginally attached to the labor force in April, 675,000 more than a year earlier. These individuals wanted and were available for work and had looked for a job sometime in the prior 12 months. They were not counted as unemployed because they had not searched for work in the 4 weeks preceding the survey. Among the marginally attached, there were 740,000 discouraged workers in April, up by 328,000 from a year earlier. Discouraged workers are persons not currently looking for work because they believe no jobs are available for them. The other 1.4 million persons marginally attached to the labor force in April had not searched for work in the 4 weeks preceding the survey for reasons such as school attendance or family responsibilities.
Table A-13, attached to the April Dept. of Labor report, indicates that only 5,935,000 (including the 2.1 million marginally attached) persons in the labor force age group but not in the labor force actually desired work. This was an increase of 121,000 from March. Thus, approximately 75 million, according to the Dept. of Labor, do not want to work. Some may be full time students, some may be retired, some may suffer from disability and some may be economically dependent on a spouse or significant other. Some may be working in the underground economy, “off the books”. Among these 75 million, it is possible that people are falling through the cracks in significant numbers. There may be many people in this group who want to work but are not being included in the labor force.
Alternative Unemployment Measurements
The Dept. of Labor reports alternative measurements of unemployment, including U6, an estimate of unemployment and under employment. It is defined as:
Total unemployed, plus all marginally attached workers, plus total employed part time for economic reasons, as a percent of the civilian labor force plus all marginally attached workers.
In April, this was 15.8%. Many think this is a better indication of employment conditions than is the official unemployment rate.
Others have speculated about how many of the people not in the labor force are classified that way, but shouldn't be. These are people who have stopped looking because of acceptance of permanent discouragement, but still desire to work. Include these people and unemployment estimates range from the high teens to the mid-twenties (percent). For an example, see Shadow Stats.
Summary
The official unemployment numbers are not bogus, but they are subject to debate about definitions, large sampling uncertainty in the headline number and argument regarding interpretation. The biggest questions arise from the discrepancy between how many initial unemployment claims are filed each month and the official numbers of increased unemployment and employment for the same month. In April, this analyst can not account for approximately 2.5 million people who filed unemployment claims and do not appear to show up in the official employment/unemployment estimates. This is apparently the approximate number of people who returned to employment during the month.
Because of large sampling error uncertainty each month, it would be better to use multi-month averages to track the numbers of employed, unemployed and people not in the work force. Although this would delay the interpretation time for the information, the uncertainty would be reduced.
It is possible that there is enough “underground” employment (off the books) that the health of the economy, as indicated by employment numbers, is not described as well as it could be. This may be particularly true for people moonlighting while receiving unemployment benefits, so the problem is greater in times of high unemployment. From this perspective, unemployment numbers may be over estimated.
Finally, different unemployment data reports have different roles to play in understanding the business cycle. Weekly initial unemployment claims can be used as an alert for the potential of the start of a recession and as a leading to concurrent indicator for the end of a recession. The peak in unemployment is always a lagging indicator regarding the end of recession, but still has value in closing the last page of the recession book. As pointed out here, a peak in insured unemployment (published with the weekly initial unemployment claims report) is a much timelier indicator that a recession is ending. Also, it was determined here that employment information useful in making strategic investment timing decisions coming out of recessions does not include any of the statistics we have found reason to question in this article.
You make a very good point. I am one of those small business owners (a sole proprietor) who does not pay myself a salary. Each month that my business income exceeds expenses, I take a "bonus", but always try to retain a business account balance reserve with enough money to cover a couple of months of expenses, since my business revenue is quite irregular. The bulk of my income occurs in January, April, July and September when clients pay quarterly fees. The other eight months see much small revenues.
I am one of those people who is "not in the labor force" according to the Dept. of Labor, yet I am more than fully employed (more than 40 hours a week). My intention was to be retired but my entrepenurial efforts have kept me employed, though "not officially". I am not "off the books" because I am paying income taxes and FICA, but I am "off the record" as far as the Dept. of Labor is concerned.
I am doing what I do because it is what I love to do. I am doing it full time plus overtime because I need the income to support the life style I enjoy. But I am not "in the labor force".
I did not come across any discussion of the birth/death deflator. Can you give me a reference to start looking into it? The DOL documentation I read says that the b/d adjustment is a monthly correction to the individual industry sector employment survey results; that the corrections are reproducible each month over the years; and that, some months have subtractive adjustments and some months have additive adjustments, industry by industry.
If I have missed something, it won't be the first time. Give me a specific reference and I'll follow up.
We are talking about the same thing. You called it a deflator and in the article I discussed the adjustment. Sorry I didn't pick up on that initially.
Actually, in the April report the deflator or adjustment for the net Birth/Death of businesses (not actually measured, but estimated) was 226,000 added jobs. This has been widely discussed with skepticism by commenters here on SA and elsewhere. The DOL literature (quoted in the article) indicates that this adjustment has only a minor effect on the total monthly unemployment total. I have been unable to find exactly how it is included in that number (and the resulting unemployment rate). The first implication is that it should not have any effect since the total unemployment is presumably derived from the household survey and the b/d adjustment is presumably applied to the establishment survey data. However, from the Q and A quoted in the article, it is clear that there is some connection (hiddeb to me at this point).
While some of the DOL literature is clearly written, other parts are cloaked in beaurocratic obscurity. Exactly how the b/d adjustment affects the total unemployment estimate is one of the areas so cloaked.
The unemployment numbers I have found most useful in determining when recessions may be ending and in making market timing decisions are (1) weekly initial unemployment claims, (2) insured unemployment total and (3) manufacturing hours worked. Links to these studies are given in the article and repeated here for convenience:
www.thestreet.com/stor...
www.thestreet.com/stor...
These reports are not affected by the measurement uncertainties discussed in the article and comment stream.
As with all Statistical Metrics I view them more as a "Weather Indicator" rather than a "State Indicator".
You Can Come To Any Conclusion If You Limit Your Data To What Supports Your Desired Outcome.
I think a better measure of economic health is Tax Revenues. These stats are less easy to fudge and you know that the Governmental Incentive to "Get Theirs" is acute.
Rockefeller Institute Personal Income Tax Revenue pub date may 13 2009
www.rockinst.org/pdf/g...
Things Are Not Well - Change Is Coming; Like It Or Not.
The Birth/Death adjustment is only a small part of the way that the BLS estimates job creation. The major method is a function that imputes new job births from the number of job deaths. They assume that non-respondents in the sample, some of whom have gone out of business, are similar to those who do respond. This has proved to be an accurate first estimate over the years. The B/D adjustment is a small tweak.
The actual job creation is on the order of 100,000 for every business day, some from existing firms and some from new ones. This has been verified by the BLS Business Dynamics series which looks at actual state employment records. It is not a survey or a projection.
I went back to the last recession and checked out the level of job creation. I reported about it here: oldprof.typepad.com/a_...
Here's a current article I'm still scatching my head about. Only in CA.
www.latimes.com/news/l...
The open secret to the unemployment shell game (moving numbers around and loosely defining terms to minimize the perception of the problem) is clearly seen in the 75 million plus that ‘are not actively seeking work’.
Within these ranks are the ‘discouraged’ workers who have not ‘actively sought employment’ in the last 4 weeks… Analysis of official unemployment figures is reliable only when delving into this massive number.
This article doesn’t even question the official line that discouraged workers constitute a minor total. What a joke.
small business drives capital creation and real job growth, and small businesses are not expanding in this negative legal, regulatory & tax climate.
Q: Do the birth/death factors vary from first preliminary to final estimate?
A: The birth/death factor for a given month does not change between 1st preliminary, 2nd preliminary, and final sample-based estimates.
Q: Are birth/death factors seasonally adjusted?
A: No, they are calculated using population data that is not seasonally adjusted and the factors are applied to the sample-based not seasonally adjusted estimates. Months with generally strong seasonal increases such as April, May and June generally have a relatively large positive factor. Conversely, months with overall strong seasonal decreases, such as January, generally have a relatively large negative factor.
Q: Can I subtract the birth/death adjustment from the seasonally adjusted over-the-month change to determine what it is adding to employment?
A: No. Birth/death factors are a component of the not seasonally adjusted estimate and therefore are not directly comparable to the seasonally adjusted monthly changes. Instead, the birth/death factor should be assessed in the context of its effect on the not seasonally adjusted estimate.
Q: Can BLS provide an estimate of the contribution of the birth/death adjustment to the seasonally adjusted monthly payroll change?
A: BLS does not calculate an estimate of the seasonally adjusted contribution of the birth/death model. The sample, the imputation of business births using deaths, and the net birth/death model are all necessary components for obtaining an accurate total employment estimate. The components are not seasonally adjusted separately because they do not have any particular economic meaning in and of themselves.
Q: How frequently are the birth/death figures revised and why do the values change?
A: Birth/death numbers are revised once a year with the benchmark revision. There are two reasons that the birth/death values change with the update. First, each year another 12 months of data are added to the historical series used in fitting the models. As a result of the additional data, the models provide updated results. Secondly, in general the amount of birth/death required is of a larger magnitude the further the reference month is from the benchmark. As a result one model is used to provide birth/death values for the first 12 reference months after the benchmark month, and the second model is used for providing birth/death values for 13 to 21 months from the benchmark. As an example, the first preliminary estimate for June 2003 was produced on the March 2002 Benchmark using a second year model. When the March 2003 Benchmark was implemented, the June 2003 birth/death factor came from a first year model.
Q: How long has CES been using the birth/death factors and is the history of birth/death available?
A: Implementation of the birth/death factors was associated with the implementation of a new probability-based sample design and estimator. The new methodology was phased in gradually under the following schedule: Beginning in June 2000 (with March 1999 Benchmark), Wholesale Trade on an SIC-basis was under the new methodology; in June 2001 (with the March 2000 Benchmark), Mining, Construction, and Manufacturing, on an SIC-basis, were added; in June 2002 (with the March 2001 Benchmark), all of Total Private with the exception of Services were under the new methodology; with the conversion to NAICS in June 2003 (with the March 2002 Benchmark), all of the Total Private industries were produced under the new methodology.
Historical Birth/Death factors are available at bls.gov/ces/cesbdhst.htm.
Q: Were estimates purely sample based before 2003?
A: No, prior to the implementation of birth/death modeling, CES used a technique known as bias adjustment to account for business births, business deaths, and other limitations of the survey.
• In 2008, the CES sample includes about 150,000 businesses and government agencies drawn from a sampling frame of Unemployment Insurance tax accounts which cover approximately 390,000 individual worksites. The active CES sample includes approximately one-third of all nonfarm payroll workers. The sample-based estimates are adjusted each month by a statistical model designed to reduce a primary source of non-sampling error which is the inability of the sample to capture, on a timely basis, employment growth generated by new business formations.
• There is an unavoidable lag between an establishment opening for business and its appearing on the sample frame and being available for sampling. Because new firm births generate a portion of employment growth each month, non-sampling methods must be used to estimate this growth.
• Earlier research indicated that while both the business birth and death portions of total employment are generally significant, the net contribution is relatively small and stable. To account for this net birth/death portion of total employment, BLS uses an estimation procedure with two components: the first component excludes employment losses from business deaths from sample-based estimation in order to offset the missing employment gains from business births. This is incorporated into the sample-based estimate procedure by simply not reflecting sample units going out of business, but imputing to them the same trend as the other firms in the sample. This step accounts for most of the net birth/death employment.
• The second component is an ARIMA time series model designed to estimate the residual net birth/death employment not accounted for by the imputation. The historical time series used to create and test the ARIMA model was derived from the UI universe micro level database, and reflects the actual residual net of births and deaths over the past five years.
• The net birth/death model component figures are unique to each month and exhibit a seasonal pattern that can result in negative adjustments in some months. These models do not attempt to correct for any other potential error sources in the CES estimates such as sampling error or design limitations.
• Note that the net birth/death figures are not seasonally adjusted, and are applied to the not seasonally adjusted monthly employment estimates to derive the final CES employment estimates.
It had a +100K adjustment in 2008 and a + 127K adjustment in 2007 in Construction. I would think that would be a modeling error.
On Jun 04 01:27 PM Market Sniper wrote:
> What I find disturbing is that the model does not account for changes in the business cycle. While perhaps valid in a growing economy, I highly doubt it is in one that is not growing. To use an unchanged adjustment model for over 50 years must be viewed with a high degree of skepticism.
On Jun 04 01:35 PM John Lounsbury wrote:
> Market Sniper - - -
>
> The unemployment numbers I have found most useful in determining
> when recessions may be ending and in making market timing decisions
> are (1) weekly initial unemployment claims, (2) insured unemployment
> total and (3) manufacturing hours worked. Links to these studies
> are given in the article and repeated here for convenience:
> www.thestreet.com/stor...
>
> www.thestreet.com/stor...
>
> These reports are not affected by the measurement uncertainties discussed
> in the article and comment stream.
There is truth in the idea that many people on benefits do work here and there are large numbers of unofficial workers, but the payrolls gives a better idea of work figures and I consider 250k increase per month to be necessary for employment to be stable, given increase in population.
In my view, all governments mess with the statistics and it's always better to be "optimistic" with economic numbers.
In the UK, they pushed people onto disability benefits (over 50 with a bad back? ) which took them off the unemployment list and reclassified as unfit to work, the danger is the damage this does to people's confidence, as they simply feel marginalized.