Comparing Value-at-Risk to Crash-and-Burn

Lots of people are newly smart and chattering away about the essential stupidity of value-at-risk modeling. Oh, those dumb financial engineers, they keep saying, How stupid do you have to be to imagine that the real world would conform to a mere formula? Ho-ho.

The trouble is, value-at-risk -- a statistical measure of financial losses your portfolio faces given various holdings and market conditions -- actually worked well for some time. It isn't that it was dopey and didn't work; it's that it made some logical and theoretical sense and approximated the real world fairly well -- right up until it stopped working.

We need to stop pretending to be so smart now, and think more clearly about what we propose to do to avoid repeating the mistakes of the past. And I hope the answer isn't that we'll do no more risk modeling, because that's dumb; and I also hope the answer isn't that we plan to get much smarter about how to measure risk, because that's naive (and an academic project). Instead, we need to think about the relationships that do work, and understand when they stop working, and what to do then (other than panic).

Put differently, we need to understand the boundary conditions of what we know. We have a financial data set, one bounded in both time and circumstances, and when we venture outside it nothing can happen (we get lucky and/or our model is more useful than we knew), or bad things can happen.

Sociologist Diane Vaughan had a great expression for what happens when you don't know what you don't know, and you creep outside the boundary conditions without failing. She called it "normalization of deviance", the business of convincing yourself that what you used to think was outside safe conditions, was actually okay because you got away with it last time.

It's sort of like what happened with the launch of the Space Shuttle Challenger. The launch team didn't adequately understand how far outside design conditions they were taking the shuttle during that cold, early morning launch, and it led to disaster. It depends, in part, on how you look at the data, as the following two figures show. The first presents the data in chronological form, which masks that we are exiting our boundaries into bad places, while the second figure shows the problem clearly.

1) A chronological plot of shuttle O-ring damage by launch date:

2) A plot of temperature vs damage to shuttle O-rings:

Comments

[levinsontodd:]

VAR is not the problem. The problem is that people don't understand it is a model.

While VAR can approximate risk, given the normal fluctuations of the market, based on past market behavior, it can't approximate for the bad, and some might argue fraudulent investing decisions that are at the heart of this financial crisis. That's because no economic or mathematical model can predict human behavior.

If you think because your portfolio meets some arbitrary level of VAR that means you are fine, then you are in trouble. If you realize it is one piece of data that can help you understand the whole picture, then you are using your brain.

It worked for a long time, because it did exactly what it claims to do: measure the value at risk in your portfolio given normal market conditions.

2008 Oct 06 11:53 AM

[Christopher...:]

The problem is that saying you're measuring "risk" based on "normal market conditions" is circular reasoning. The major risk rears its head when multiple things go wrong simultaneously, and given the number of different things that can go wrong, it'll inevitably happen.

Those are precisely the times when LTCM or the CDO market rolls over and dies, and simply assuming those situations out of your model means that your model is inadequate for the specific job it's supposed to do.

2008 Oct 06 12:33 PM

[whidbey:]

We need to stop and think more clearly about what we propose to do to avoid repeating the mistakes of the past. I think you are in error.

Since this is not a physics problem, but a behavioral finance issue, it is not probable that the same mistakes will be committed again, but rather, some variation with sufficient change to rule out the lessons learned. Humans err not in patterns, but in knowing the facts and risks associated with an event. They "play hunches". The cost of knowing/learning about these changes is often not economic,
so we do not exhaust research on each variation of past errors.
In short learning concrete lessons is typically to misinform future decision making. I think you are wrong.

2008 Oct 06 02:12 PM

[User 265729:]

Whidbey:

It's not probable?

It IS happening again as we speak! We are following right down the footsteps of the late 1920's, step, by step!

Whidbey quote:

"Since this is not a physics problem, but a behavioral finance issue, it is not probable that the same mistakes will be committed again, but rather, some variation with sufficient change to rule out the lessons learned."

2008 Oct 06 04:17 PM

[Smarty_Pants:]

My guess is that current risk models accounted for "probable" variations and did not take into account 'worst case' risk. If they had, they never would have leveraged beyond 1-to-1 with their debt. This can be handled with sensible internal controls.

More difficult is the conterparty risk assessment. How do you know that the guy who just signed a deal with you is leveraged 100-to-1, might go under, and leave you holding the bag?

i.e.
How did AIG ever make CDO 'insurance' contracts paying out vast multiples of their net equity on nearly identical underlying items? Assuming that only a small percentage of these would go bad at any one time was a huge mistake that cost them everything. This is anything but diverifying, it is concentration into one basket with huge leverage. They were asking for trouble, and they got it in spades.

Same with Fannie/Freddie and Alt-A, subprime, etc.

2008 Oct 06 04:55 PM

[Mbuna:]

The assumption that clear headed thinking can prevail in an asylum full of financial lunatics leaves me wanting. Okay, okay, they're financial deviants not lunatics. Yes, if people paused to think before the feeling of just wanting that little more overtook them it could do some good, But that is certainly NOT the Wall Street of these past few years and at the very least some parental controls are called for and even possibly some electroshock therapy to cure that deviant behavior.

2008 Oct 06 10:57 PM

[Saul Albom:]

blog.saulalbom.com

2008 Oct 07 04:47 AM

[Saul Albom:]

Aristotle and Plato both knew he who controls the educational system controls the society. Finance professors and risk managers still teach students that human events occur in normal distributions. They teach that through the past we can predict the future with great accuracy. Students are taught mathematical tools such as linear analysis, correlation and beta to analyze the past and plot the future. Many phenomena can be predicted via normal distributions but NOT human interactions.

Often financial experts visualize risk or events a sign wave, reasonable normal and predictable. Events get worse, things get better, prices (phenomena) fluctuate about their mean and always revert towards it.

One of the massive fallacies of normal event distributions is the mitigated chances of extreme events. Before this year one could have asked what was the probability of our major financial system crashing? What were the chances of the major investment banks with the most brilliant financial professionals in the world going under? What were the chances of real estate crashing, globally? (The second fallacy of normal distribution is that extreme events are more severe then could possibly be predicted often to the order of thousands or millions greater then a normal observation)

What is ironic is blindness and faith that the highly unlikely can not happen only serves hasten them and make these extreme events more probable and worse. Take the billions of credit derivative swaps that were written on sub prime debt.

Sub prime means that the borrower could not afford the mortgage. These loans were only made because it was assumed that housing prices would continue to go up 10% a year into infinity. The loans then would eventually be re-financed and every one would be happy. The invester got paid. Hard working people who could not afford nice houses would now have one. This was win-. However it was inevitable that there would be a correction, people can only afford housing up to a certain portion of their wages. Finance professionals are taught linear analysis or other mathematical tools of a similar brand that pre-suppose that the future will behave like the past.

We must humble ourselves and realize we can not predict complicated phenomena such as stock price, earnings, geo-politics, heck even the weather! The mind may be master of the world but not the future.

Lehman brothers had the largest bankruptcy in history with 640 billion in debt. In their hubris they sought to leverage up to 40x and greater. Long Term Capital had in excess of 100x leverage! In their minds they could borrow and invest safely. They could know. If one can know the truth about something and know the truth about a system, then you can predict the future of that system. If a finance guy can know the future then he can make money, he can make a lot of money. So investment banks thinking they could know and control leveraged up magnifying their gains. In the end they did not know the truth and were crushed.

Let me state that their gambles were good for a long time and printed money. However, their understanding of financial phenomena was a very close approximation, but not the real thing. As they made more and more money they leveraged more and more and made an other linear mistake: because we never lost big before, we wont lose big in the future. We know what we are doing.

Before this week truth was: of course Lehman, Bear Sterns and Merrel can leverage 40x, they are brilliant, they have statistics and risk managers that can protect them. But they were unable to predict extreme events such as the real-estate crash. Now we look at a company with 40x leverage and think they are insane. Today Goldman has 20x leverage. Are they insane, have hubris or brilliant?

There needs to be a complete revolution in risk management. Something needs to happen like "efficient market theory". It is our lack of understanding of risk when it comes to human events that lead to this financial catastrophe. Even in statistics it is normal to throw out "outliers". One extreme event can destroy an entire portfolio or company. It is unknown how this works. There needs to be more studies of extreme events, not normalcy.

Again, the major reason why we are having the worse financial crisis since the great depression is because major players thought they understood the nature of risk and extreme events, that they could predict and know the future thus giving them the confidence to massively leverage. Until the idea of control is broken then we will not learn from our mistake.

2008 Oct 07 04:47 AM