Household vs. Establishment Surveys: Which One Matters

by: Barry Ritholtz

For some reason, there seems to be this ongoing wishful thinking that somehow, this economy is creating far more jobs than we are measuring.

I chalk it up to projection:  When one's own situation is personally improving, there is the temptation to believe everyone else is too. But that simply is not the case. The reality of the Zero Sum Game in trading markets, nature, and society as a whole, simply makes that impossible. For every leopard that eats dinner, an antelope dies; For each Amaranth that blows up, someone else is on the other side of the trade improving their performance.

Far too many people have a genteel misunderstanding of what zero sum is when applied to investing and trading, to economics and even information.

Which leads us to the Household Survey of Employment: Despite Greenspan, the BLS and other authorities repeated discussions why this is a less reliable number than Establishment Survey (NYSE:NFP), it simply refuses to go away: The differences between the Household and the Establishment (NFP) surveys is a meme that's long past its use by date. Its resurrection is like Jason in Friday the 13th or Freddy in Nightmare on Elm Street:  this one simply won't stay dead and buried.

A quick review is in order. As we have discussed all too many times, the Household survey measures:

- Agriculture and related employment;
- Uncompensated Workers;
- Part Time Workers;
- Unpaid Family Employees;
- Workers absent without pay from their jobs;
- Self employed, Work-at-home Contractors;

-- none of which are counted in the Establishment (Non-Farm Payroll) Survey. In fact, the BLS specifically looked at and compared the two data
series. Once they made an adjustment so both surveys were
counting the same thing, any gap between the two disappeared.

Over the past few years, the Household survey has become the last refuge of bullish employment scoundrels. So rather than having a fit of pique when I see the same old meme getting circulated AGAIN, I decided to have a closer look at the smaller, less reliable, not payroll tax driven but self-reported Household survey.

Let's put all that messy "factual stuff" aside for the moment. How did the Establishment NFP survey show but 51k new jobs, while the Household survey show 271k job creations?

A sharp-eyed friend directed  me to the “Explanatory Note” BLS releases with the Household Survey. Specifically, I scroll down under “-8-“, paragraph 3 “Reliability of Estimates.” This discusses the “confidence interval” a/k/a/ as the margin of error:

The confidence interval for the monthly change in total employment from the household survey is on the order of plus or minus 430,000. 

Suppose the estimate of total employment increases by 100,000 from one month to the next.  The 90-percent confidence interval on themonthly change would range from -330,000 to 530,000 (100,000 +/- 430,000).

These figures do not mean that the sample results are off by these magnitudes, but rather that there is about a 90-percent chance that the "true" over-the-month change lies within this interval. Since this range includes values of less than zero, we could not say with confidence that employment had, in fact, increased.

You read that correctly: The margin of error is 163% of the reported monthly Household survey.  This means that we a have a 90% degree of confidence that the “true over-the-month change lies within this interval" in between from -159k to +701k. 

Or as BLS specifically states: "Since this range includes values of less than zero, we could not say  with confidence that employment had, in fact increased."

Case closed.

About this article:

Want to share your opinion on this article? Add a comment.
Disagree with this article? .
To report a factual error in this article, click here