Recognizing the Fallibility of Prediction Markets

Every now and again, I read something that simply makes my head spin. This morning’s NYT article on prediction markets was just such a head-spinning column. It reached such a bizarre conclusion that it begs for comment.

Longtime readers are familiar with my views on prediction markets. I believe they have some value, when applied correctly. Where prediction markets excel is in acting as a collective real time polling mechanism for their participants. The closer the collective group is to the broader population, the more accurate these markets tend to be. Where they do poorly is when the collective attempt to “pool their ignorance” and forecast the random or the unknowable.

There is strength in any mechanism that can factor a large pool of participants' collective experience and knowledge, discount several possibilities into the most likely outcome. These market mechanisms are hardly the Oracles of Delphi their supporters make them out to be. It is only by din of Humans’ even worse forecasting abilities (with experts leading the way down) that these markets garner so much respectability in the first place. Blame fuzzy thinking for placing too much credibility on a mechanism with so mixed a track record. There is also a subtle but important distinction between forecasting the future versus discounting various outcomes.

Which leads me to today’s NYT column.. In a bizarre twist of logic, the massive failure of the prediction markets in the US 2006 mid-term elections somehow gets credit for being right. They were not a day late and a dollar short, they were completely, totally and incontrovertibly wrong.

Except in the NYTimes.

Consider the following: As of 11:50 p.m. on the evening of Election day, with the voting completed (except for Hawaii) and the majority of the ballots counted, Intrade gave the Republicans an 85 percent chance of retaining the Senate. Of course, we know that’s not at all what happened. If that’s not a forecasting failure, then what is?

If forecasting the results of an election held yesterday is considered prescience, then sign me up -- and find me a bookie who will take my Superbowl bets on the Monday after the game. I suspect I will at least cover the spread.

To reiterate my views, these markets -- thinly traded, easily manipulated, poorly administered -- do have some real value. However, they have hardly been "remarkably clairvoyant," as the Times describes them. Consider these recent acts of "clairvoyance:"

GOP Retention of Senate
Michael Jackson Trial Results
Morgan Stanley CEO Purcell resignation
Howard Dean’s Iowa Primary
pre-Election 2004
Election day trading frenzy

There are some interesting explanations for some of these errors: Pooling the collective ignorance of millions does not produce Wisdom. The Michael Jackson Trial and Morgan Stanley's CEO are perfect examples of that.

As to the prediction market's failure of the Howard Dean primary and the 2006 mid-term elections, I've been toying with a theory. Since the political affiliation of Wall Street tends toward Republican, and since there is a big overlap between Wall Street and the participants of the prediction markets, the collective had an inherent bias built into it. Remember, where these markets excel is when they act as a realtime polling mechanism of their participants. If the pool has a bias, the outcome may very well too. Hence, these wide-scale failures.

~~~

Bottom line: While having value, prediction markets are subject to error and bad outcomes. Some of this is due to their relative thinness of the markets; some is due to the inherent biases of the participants, and their failure to parallel the population at large.

They should be considered with the recognition of their fallibility.

Source:
Odds Are, They’ll Know ’08 Winner
DAVID LEONHARDT
NYT, February 14, 2007
Economix
http://www.nytimes.com/2007/02/14/business/14leonhardt.html