When Quants Fail

I always have found it astounding the confidence some people place in quantitative models. Every time there is a market crisis, news reports trickle out about high profile failures of quant models. Meriwether is but the latest.

Even more amazing is how people throw money at quant investment shops without asking questions. In a prior life, we experienced this first hand. We were, of course, old fashioned - we actually researched each company individually. We would go into client meetings and get grilled by a skeptical audience on each holding, specifically harassing us about the ones that didn't work. Then the PhD's from the quant firms would show up, and the clients would get on their knees and chant, "we're not worthy, we're not worthy..." Not really, but you get the idea.

People just get impressed with complex mathematics, especially when they don't understand it. As a trained mathematician, I often have a hard time understanding why. I don't, however, have a hard time understanding why these models fail periodically.

The models capture certain relationships between assets, markets, and other variables. A simple example is this: retail stocks tend to move together, so they are correlated. A quant model will capture that, because it happens to be true most of the time. If you want to hedge one retail stock holding, you short another or a basket of others. The problem is that it won't work every time - occasionally your stock will go down while the ones you short will go up. In addition, when you most need the model to work, it will likely fail. In the case of long-short pairs, if you structure your portfolio correctly, then a model failure on one pair won't destroy your wealth. Sensible quant analysts understand this.

But the asset markets are anything but sensible. There are a lot of really smart people with powerful egos and huge vested interests in their way of doing things. A lot of effort goes into "model mania", and some believe they can model away the risk.

The simple truth is this: quant models use statistics and probability. Probability simplifies reality, but it does not explain reality. In the case of a card game or a dice roll, explanation is unnecessary - we don't need to know the physics behind the dice roll that caused it to land exactly as it did. Markets, however, are significantly more complex than cards or dice. In addition, the data used to make the models is significantly less reliable. Therefore, the models will fail from time to time.

Why do they fail so spectacularly in a crisis? Usually it is leverage. Some very smart fools believe that they could eliminate risk and therefore magnify return.

What does this mean for investors? Quant models can be useful, but they are not a replacement for good judgment. Leverage, real or implied, does not increase return without a commensurate increase in risk. Leverage does not take skill. Always consider all of the risks when making an investment, not just the statistically measurable ones.

Note: the title, "When Quants Fail" is a take-off of Lowenstein's book on Meriwether and LTCM, "When Genius Fails". In it, he quotes GK Chesteron: "Life is a trap for logicians because it is almost reasonable but not quite; it is usually sensible but occasionally otherwise: It looks just a little more mathematical and regular than it is; its exactitude is obvious, but its inexactitude is hidden; its wildness lies in wait."

Comments

[cfish]

Amen. Some of the most aggresive trading strategies out there are nearly completely models. It's only been a decade since LTCM and people forgot about how dangerous blind trust in quant models can be.

Having met some of the fixed income quant people at Citi, I can't say I feel bad for the bunch.

[nickgogerty]

a model by definition is wrong. It is understanding how it is wrong that makes it interesting.

[nickgogerty]

I would suggest using gott's theorom to predict when the model will or condition will fail. All things change asking when is an insightful question. try it nickgogerty.typepad.co...