Beware Economic Forecasters

Whether or not ‘Green Shoots’ are in fact appearing with the U.S. recession entering its 20th month, the press continues to report on the economy more and more these days. The stories may generally seem to have a more negative tone, but the press is always concerned about the economy – in good times and bad.

Fortune magazine’s recently developed Big Picture Index is an economic indicator that is supposed to predict the end of the recession. On a scale of 0 to 7 (0 being the weakest), the index stood at 1.5 this week.

But the team at stock market research firm Birinyi Associates isn’t buying it. They point out that Fortune failed to provide a robust history of the index, only showing the past 52-weeks worth of data, despite the fact that six of the underlying components of the index have long histories. While it was able to find the underlying components of the Big Picture, Birinyi couldn’t replicate the index data provided by Fortune from May 2008 to the present.

The firm told clients:

We are hard pressed to explain how an index made up of just seven members can fall 72% when the worst component is down 85% and two of the seven are up and the remaining four did not fall anywhere near 72%.

Birinyi also picks on the Wall Street Journal and its monthly survey of economists, which is currently forecasting GDP to turn positive starting in the third quarter. The firm said the accuracy of these forecasts, which began in 2003, leaves something to be desired.

For example, with two weeks left in the fourth quarter of 2008 the survey forecast a 4.3% decline in GDP for the quarter. The actual result was a 6.7% decline, which works out of a difference of 47%, Birinyi noted.

Furthermore, examining the average forecast vs. the actual results since 2003 we find that the closest the survey has been to the actual was the third quarter of 2005 but in most instances they were well wide of the mark.

But the media isn’t the only target of this forecasting attack. Birinyi also goes after the Federal Reserve Board, which is required to provide a forecast twice a year and has been forecasting GDP since 1992.

The Fed’s forecasts have been widely inaccurate with discrepancies of as much as 2.4 percentage points, Birinyi said. Prior to 2000, the FRB was consistently too negative, while it has been too positive after 2000, the firm added.

The lesson is one you should have already known: While there are many analysts willing to offer their opinions on the economy, their records are far from perfect and the forecasts should be taken with a grain of salt.

Comments

[morph366:]

You point out that-
"Prior to 2000, the FRB was consistently too negative, while it has been too positive after 2000, the firm added."

This suggests that the bursting of the Nasdaq bubble really did scare the Fed and helps to explain the Greenspan put.

Jul 05 11:53 AM

[reluctantQuant:]

Humans stink at forecasting. Always have. Wish we didn't, but we do... direction is a bit easier, but the time component - lousy.

Sure, I can predict when the apple will hit the ground, if I know when it leaves the tree. Gravity is awfully consistent. Not too many forces or unknowns at work there. Ah.. you want to predict when it will leave the tree? hmm.... I'll make a better spreadsheet.

How many katrillions of dollars have been spent over centuries to predict the weather? Now, there's a nice complex system to discuss, much like global financial markets. But humans have been working on this weather forecasting gig for MUCH longer. How's that going? Yep, after a few katrillion dollars, we're good to forecast a solid three DAYS forward. A "fifth day" forecast has no utiility, and we all know this.

Good investing methods must EXPECT forecast error and adapt to changing conditions, just like a good sailor or mountaineer. Black Litterman asset allocation methods actually take a good approach to this (another story).

Go back and look at any economic article from 2006. Yes, there's that precious 10% that "got it right" during any given month, but very very few show any persistence over time. And just how does one know when that 1 out of 100, the contrarian of the hour, happens to be right - at that moment? Ya don't.

Forecast, schmorecast. Agility and prepared-ness beat 'em every time.
--rq

Jul 06 09:02 AM