Four Things You Didn't Know About Unemployment

There is a lot of commentary about the unemployment rate. Various pundits are determined to find the worst possible interpretation of data, and also to impugn any official reports. Let us take a closer look. Many of the answers come from the following source, helpfully highlighted by Barry Ritholtz at The Big Picture:

Measuring Unemployment in the Nineties
Janet L. Norwood and Judith M. Tanur
The Public Opinion Quarterly, Vol. 58, No. 2 (Summer, 1994)

Anyone who took a few minutes to read this important article would have some real insight into the world of policy nerds -- people like us!

Since few will be willing to invest the time to do a little real homework, here are the key things you would learn.

#1. Presidents do not dictate changes in statistical procedures.

This flies in the face of the popular "rogue economist" selling his conspiracy theory. It will also seem implausible to the "B" movies fans. Nevertheless, it is the boring truth. The Clinton-era changes in employment measures were the result of a long process, thoroughly examined and challenged by many groups. The final measures adopted the recommendations of the 1979 Levitan Commission, chaired by the renowned labor economist, Sar A. Levitan. Here is a glimpse of the process, taken from the Norwood and Tanur Article:

The Process
The leadership of the two agencies [Census and BLS, jm] began a joint effort to develop a comprehensive strategic plan to reaffirm the CPS's leadership in labor force surveys by using state-of-the-art questionnaire design and advanced technology. A series of meetings with the senior staff of the two agencies, including the BLS commissioner and the Census director, delineated the comprehensive set of tasks that needed to be carried
out. Several joint task forces were established to develop the research issues that needed to be addressed and to establish the necessary plans, budget estimates, and time tables for implementation.

The work began with efforts to identify areas of the old questionnaire that might need change. First, experts from BLS and the Census, as members of the CPS Questionnaire Redesign Task Force, reviewed problem areas. Based on their experience in developing and analyzing the data, they were often able to see questions that had a high potential for response error or that failed to implement existing definitions properly.

They also noted questions needing definitional changes and identified areas of labor force analysis for which more data were needed. They believed that because of the dramatic increase in labor force participation of women and the rise in part-time and temporary workers, more information was needed from the survey about these categories of workers. Bureau of Labor Statistics staff were also eager to develop a more user-friendly processing system so that these and other important analytical issues could be addressed more easily and more quickly than in the past.

Next, the task force recommended a program of preliminary research. Respondents were brought together in focus groups to explore their understanding of the concepts underlying the questions asked. It was this technique, for example, that confirmed a discrepancy between the respondents' and the CPS's use of the term "on layoff."

Another preliminary technique tapped interviewers' experience with
other CPS concepts by using card sorts to explore their coding of
open-ended questions about job search and other matters (Fracasso 1989). The next step was to design two new questionnaire....

This goes on for pages. Any objective reader of this report will immediately dismiss the conspiracy theory. There is no way that all of these academics, bureaucrats, and observers would have maintained some conspiracy. They have different ideology and policy preferences. What they shared was a desire to "get it right."

#2. The changes made the employment measures worse. Here is a key quotation:

In November 1993, government information suggested that
the jobless rate, undoubtedly one of the most politically sensitive of
all federal statistics, had been substantially underestimated-by about half a percentage point-for many years. And this, in spite of the fact that the Current Population Survey (CPS), the 53-year-old survey that produces the data to assess the unemployment rate and provides a great deal of other labor force information, has long been used as a model of the best in survey design both in this country and around the world. What had changed?

The biggest factor was the increase of women in the labor force. The BLS statisticians, like all data lovers, want to see continuous series, unchanged by methodology. When the world changes, sometimes the questions must follow. The BLS altered the workforce definition in a way that instantly increased the most common unemployment measure --- because it was the right thing to do.

Part of the change was a clarification of what it meant to be part of the work force. This list of questions determines what respondents were really doing -- looking for work, attending school, keeping house or going fishing. The current measures of labor force participation resulted from this work. In prior years, we have no idea of what the answers might have been.

#3. The Federal Government measured the impact of the changes.

The BLS and the Census Bureau did an "overlap" sample where both the old and new questions were asked of 12,000 respondents. By comparing these answers, one can infer the impact of the changes.

It is pretty obvious that this thoughtful statistical technique is the work of scientists, trying to find the right answers. Those who are making accusations about political motives need to explain why this extra care was taken.

#4. Current unemployment, while very bad, does not approach that of the Great Depression.

Here at "A Dash" we have a mission of helping investors to find the best information from the best sources. Often this means a focus on the change in economic conditions. Some source seem determined to find the absolute worst interpretation of every economic report. This is currently a great way to get popularity on the Internet. Some others seem to have a bullish spin on everything, perhaps going for a niche market.

Our idea is a bit different. We try to find the best indicators from believable sources. We classify these indicators as leading or coincident. We then stick with the program, unlike others who discard or change measures in a heartbeat when they no longer like the results.

The Great Depression comparisons are erroneous. They also seem quite unhelpful in recognizing key economic trends. Here is a source for thoughtful readers: This Is Not Another Great Depression.

In fact, there are no continuous unemployment sources showing that current results are comparable to the Great Depression. Anyone suggesting otherwise is blowing smoke.

And that is the official answer to our Summer Quiz Question #6. This is the last of the answers, and we will declare a winner this weekend, awarding a nice prize from Amazon.com. In practice, anyone who did the homework and got most of these answers right enjoyed the summer rally, and ignored the "sell in September" rhetoric."

Comments

[Mad Hedge F...:]

ubg With the collapse in employment, it is all about “location, location, location.” No surprise then that Michigan leads the country with a 15.2% jobless rate caused by the implosion of the auto industry. Detroit is suffering a horrific 17.3% rate. Nevada came next at 13.2%, blasted by both barrels of a shotgun by simultaneous layoffs in housing and hotels. Rhode Island at 12.8% took big hits in health care services and tourism. My home state of the “Land of Fruits and Nuts,” California, now has a 12.2% jobless rate, the worst in 70 years. Housing was the grim reaper here, and now 20,000 teachers have started collecting unemployment checks, thanks to our bankrupt state government. Who has the lowest unemployment rate in the nation? Sunny North Dakota at 4.3 %, where healthy agriculture and energy industries kept people in work, and ranking 48th in population, have almost no people to fire anyway. It also helps that the “Roughrider State” was completely bypassed by the subprime boom, and for many years was the cheapest place in the country in which to own a home. To get unemployment back to a normal 5% rate, Obama has to create 20 million jobs by the next election in 1012, one tall order.

Oct 01 06:42 AM

[Carlos Lam:]

What you write may be correct, but it is always wise to view statistics and ask whether the provider somehow has an incentive to skew them. Any administration has the incentive to skew unemployment lower to attempt to make things look better. The same goes for GDP figures.

Again, you may be right. The question that we have to ask ourselves is: How much do we trust political animals to produce accurate data? My answer is: not much. The political animals do not have the price system that is required to ensure that their data is accurate. Private data providers (i.e. ADP, TrimTabs) produce data for profit; they either prosper or fail based on the quality of their data. I trust their figures more.

Oct 01 07:17 AM

[Old Trader:]

Jeff,

Thanks for an informative article, but a question. From what I read, one of the aspects that's most questioned by those critical of the way employment/unemployment statistics are compiled, is the birth/death component, and your article makes no mention of it. Any particular reason?

Oct 01 09:08 AM

[Hmm?!:]

This article is very weak.

#1 Presidents do not dictate statistical procedures?
Anyone who has studied Internal Audit will tell you the NUMBER 1 factor in any large scale fraud is "Tone at the Top". WorldCom, Enron, Nortel, Adelphia, Tyco. In each case thousands dictated to, by a handful of corrupt at the top.

#2 A clarification was made.....OK however you provide no data to show changes made any numbers worse, or even produced a significant result......... if the changes did anything at all to the results.

#3 We measured the impact. OK...... and any results? And what would you expect the government to say???......Yes we made changes but No we never looked at the impact!?

#4 Once again....... any numbers....... because your embedded link leads to nowhere.

This article appears to be unsubstaniated opinion and fluff.
"In God We Trust, all others must bring data" Deming

Oct 01 09:23 AM

[John Lounsbury:]

Jeff - - -

This is a well researched article. The Norwood and Tanur article is a valuable research reference. I was not aware of it until you pointed it out. An area that I read carefully was the discussion of the DOL questionnaires. One of the things I have thought would be valuable to add would be age demographics. My work on the decline in the labor force could be much more illuminating if I knew the structure of the population not in the labor force. For example, a 30 year old person who is counted as "not in the labor force" has a much higher probability of returning to the labor force in the future than a 60 year old.

After considerable research, I have become convinced that the BSL is professionally run and free of political influence. That does not mean that serious people can not criticize how their data is collected and how it could be better used. I have made some suggestions in those regards, but I have never found a basis for suggesting that "propagandists" need to be corrected.

Thanks for writing on this topic. Some do not appreciate your efforts, but others do. Count me among them.

Oct 01 09:49 AM

[John Lounsbury:]

Sorry for the typo - the acronym should be BLS (Bureau of Labor Statistics)

Oct 01 09:52 AM

[Jeff Miller:]

The last link "This is NOT another great depression" should be as follows: freakonomics.blogs.nyt.../

Sorry for the inconvenience.

Oct 01 10:01 AM

[Jeff Miller:]

Carlos -- I agree with your point about looking at the incentives of data providers. As someone with special experience in this field, that is how I am trying to help readers.

The problem is in thinking that "the Administration" controls what is reported in most data releases. That assumption is incorrect. Most of the work is done by professional economists and statisticians who keep their jobs even as Presidents come and go. I covered that extensively in my article, "A Crib Sheet for Government Data" oldprof.typepad.com/a_....

Take a look there, and you will get some ideas both about data sources and the "spinning" of data from others.

Thanks for the good question.

Jeff


On Oct 01 07:17 AM Carlos Lam wrote:

> What you write may be correct, but it is always wise to view statistics
> and ask whether the provider somehow has an incentive to skew them.
> Any administration has the incentive to skew unemployment lower to
> attempt to make things look better. The same goes for GDP figures.
>
>
> Again, you may be right. The question that we have to ask ourselves
> is: How much do we trust political animals to produce accurate data?
> My answer is: not much. The political animals do not have the price
> system that is required to ensure that their data is accurate. Private
> data providers (i.e. ADP, TrimTabs) produce data for profit; they
> either prosper or fail based on the quality of their data. I trust
> their figures more.

Oct 01 10:09 AM

[Jeff Miller:]

Old Trader -- The birth/death adjustment is the focal point for a misguided attack on the BLS. I have written extensively on this topic, but the best way to check it out is my recent three-part series. Here is the concluding article, which has links to the prior two and also to other sources. oldprof.typepad.com/a_...

The data and results from these articles has not been challenged. B/D critics just say the same thing every month, never looking at research data to see if they were correct.

Thanks for helping me to highlight this.

Jeff


On Oct 01 09:08 AM Old Trader wrote:

> Jeff,
>
> Thanks for an informative article, but a question. From what I read,
> one of the aspects that's most questioned by those critical of the
> way employment/unemployment statistics are compiled, is the birth/death
> component, and your article makes no mention of it. Any particular
> reason?

Oct 01 10:16 AM

[Tom Armistead:]

Jeff, it always surprises me that unemployment is developed by surveys, although thinking about whether someone is actually looking for work it does seem that the only way to find out is to ask people.

Conspiracy theory has a place in this world, but BLS unemployment statistics is not it. The effort and indignation would be far better spent on the relationships of the SEC, big banks, politicians, etc., stuff like CDS and naked short-selling. It's a huge field, so many fruitful areas, why waste the effort on unemployment statistics?

Historical comparisons seems like the most fruitful area for investors who are trying to understand the interplay between the economy and equity prices. That takes us back to the early 80's.

This morning I was on initial claims, computing them as a percentage of the workforce. The early 80's were worse than any point during 2009 on that basis. 2009 is worse on ongoing claims as a percentage of the workforce, it is harder to find a job if you lose one.

Or maybe people in their 60's, after they lose real jobs, are very slow to go down to Walmart and apply for work as greeters.

Oct 01 11:05 AM

[thiazole:]

Good article. Thank you for your efforts here.

The reason I've always been skeptical of the conspiracy argument is how difficult it would be to pull off. If the data just has an optimistic bias, then it does so during the good times as well, so everything is still relative.

But if the government was going to perform outright fraud to paint a rosier picture, why even report data that confirms the worst recession since the 30s? Why not make it look more like a moderate recession instead of one so severe? If they were going to commit fraud, I'd think they'd do a much better job of it. Besides, making everything look rosier than it really is makes it harder for Obama to sell all of his pet social projects. If there was fraud, it would make more sense to create a fake recession in order to give the government more power than to fake that a recession is more mild than reality. Besides, by reporting a recession to be worse than it really is, you end up with a surplus of positive economic data that could later be used to fake the recovery ie, if you are reporting 10% unemployment, but it is really 8%, at any time the you could adjust the numbers back to reality and say it was an improvement caused by one of your pet social projects.

Oct 01 11:51 AM

[enigmaman:]

Im not one who believes that even this Admin would contemplate committing fraud, but I do believe deals have been struck with certain bankers and financial institutions, where all agreed that Gov would stop demagoguing these businesses as it did relentlessly as the market fell, the Admin would get out of the way so the WS bigwigs could work their magic to prop up the markets, which would improve consumer confidence, 401K, world view and last but not least help those companies bailed out by the Gov repay their loans with interest. We will never know what went on behind closed doors when all these guys met but one thing I do know Obama didnt bad mouth any of them once the meetings were over. You believe what you want Ill believe what I want.


On Oct 01 11:51 AM thiazole wrote:

> Good article. Thank you for your efforts here.
>
> The reason I've always been skeptical of the conspiracy argument
> is how difficult it would be to pull off. If the data just has an
> optimistic bias, then it does so during the good times as well, so
> everything is still relative.
>
> But if the government was going to perform outright fraud to paint
> a rosier picture, why even report data that confirms the worst recession
> since the 30s? Why not make it look more like a moderate recession
> instead of one so severe? If they were going to commit fraud, I'd
> think they'd do a much better job of it. Besides, making everything
> look rosier than it really is makes it harder for Obama to sell all
> of his pet social projects. If there was fraud, it would make more
> sense to create a fake recession in order to give the government
> more power than to fake that a recession is more mild than reality.
> Besides, by reporting a recession to be worse than it really is,
> you end up with a surplus of positive economic data that could later
> be used to fake the recovery ie, if you are reporting 10% unemployment,
> but it is really 8%, at any time the you could adjust the numbers
> back to reality and say it was an improvement caused by one of your
> pet social projects.

Oct 01 03:07 PM

[Carlos Lam:]

On Oct 01 10:16 AM Jeff Miller wrote:

> The data and results from these articles has not been challenged.
> B/D critics just say the same thing every month, never looking at
> research data to see if they were correct.

I have a more fundamental problem with the B/D Model: why report it at all? The B/D adjustments are not hard numbers; they are estimates (and maybe good or maybe bad). Why "clutter" the data with an adjustment that is vulnerable to political influence?

Yes, I know you will argue that the political influence is minimal, and perhaps you are correct. The very fact that it EXISTS, though, and causes readers of the data to even wonder whether the political influence is IN the numbers is problematic.

Again, I re-state that there are better providers of jobs data out there that simply DO NOT have a political agenda (i.e. ADP, TrimTabs). I would much rather use data that I know is free of an agenda versus data that I think is PROBABLY free of a political agenda.

Oct 02 10:05 AM

[Carlos Lam:]

On Oct 01 11:51 AM thiazole wrote:
>
> But if the government was going to perform outright fraud to paint
> a rosier picture, why even report data that confirms the worst recession
> since the 30s? Why not make it look more like a moderate recession
> instead of one so severe?

I don't argue that there's some huge conspiracy. The difference is at the margins. When deciding between two equal methods or figures, will a BLS economist go with the rosier one or the not-so-rosy one? I believe he'd go with the rosier one. This ins't conspiracy: it's responding to the incentive of an employee who wants to please his boss.

Oct 02 10:11 AM

[Jeff Miller:]

Carlos -- This is a very good question. Rather than answer in the context of a single comment, I'm going to make it part of an article reviewing today's report.

Thanks for following up.

Jeff


On Oct 02 10:05 AM Carlos Lam wrote:

> On Oct 01 10:16 AM Jeff Miller wrote:

Oct 02 08:00 PM