Bank Risk: Will Basel II Finally Discredit VAR?

"Risk management is a serious business. Accordingly, the production of a risk 'measure' must be subjected to the question 'how do you know what you claim to know' – in other words, epistemology." (Nassim Taleb)

Last month, a reader at the University of Michigan asked about our March 27, 2007 comment, "Countrywide Financial: It's All About Liquidity," where we asserted that "the use of risk pricing tools such as Value at Risk or 'VAR' models and other types of statistical routines arguably amplified the effect of excess liquidity, boosting the throughput of the Wall Street mortgage origination machine, generating big fees, and vastly expanding the pool of risk for end-users."

Asked the reader: "I would like to know how VAR can cause banks to take more risk than what is acceptable. Doesn't the Fed with its extremely loose monetary policy deserve more blame?"

Good question. The Fed's "easy money" monetary policy in the early part of the decade, what we've coined as the "Greenspan Effect," clearly contributed to "exuberant" behavior by investors, pushing risk spreads below the economic cost for many assets. The spectacle last year of CDS for subprime trading at annual spreads less than a quarter of the expected default rate on such portfolios or corporate CDS trading through the yields on the underlying bonds are two cases in point.

But no matter how pernicious the Greenspan Effect, in our view, the use of risk measures such as VAR contribute far more significantly to the problem. Determining "what is acceptable" is the key issue for risk managers, both in terms of setting the minimum expected loss in a given scenario and then how to benchmark these forward-looking estimates.

To review, VAR models summarize the expected maximum loss (or worst loss) over a target horizon within a given confidence interval. To us, this is an elegant way of saying "I don't know." Or to quote the author of Fooled by Randomness, Nassim Taleb: "There is an internal contradiction between measuring risk (i.e. standard deviation) and using a tool with a higher standard error than that of the measure itself."

If you have not done so, read Taleb's excellent LSE paper, "Epistemology and Risk Management."(.pdf)

The trouble with VAR models is that the methodology says nothing about specific risks regarding specific transactions, yet provides risk practitioners with the false impression that the particular risk has been measured. So widespread is the delusion that VAR is effective to measure risks of particular exposures that federal regulators are about to adopt it as the central method for measuring bank capital adequacy under Basel II (see our comments to the FDIC on the Basel II proposal by clicking here.)

By providing a technical framework for estimating market risk that, on the surface at least, has credibility, the Sell Side and the major rating agencies have evolved VAR into a powerful mechanism for increasing both transaction throughput and leverage. Simply stated, if the overall risk calculated in the VAR model appears to be low, then additional risk may be taken.

Since the Greenspan Effect pushed actual default rates on loans and bonds to near zero, particularly for mortgage collateral, VAR models became the Sell Side's best friend. By relying on assumptions of normality in the distribution of possible future events and using recent historical data, VAR models effectively minimize the true financial risk taken and thus are a key enabler of the vast expansion of leverage on Wall Street.

An interesting possibility comes into prospect, however, with the implementation of Pillar III of Basel II, the requirement for market discipline, where banks will be compelled to publicly disclose and benchmark the efficacy of VAR models against their actual results, including public "mark-to-actuals" results for VAR, Defaults, Loss Given Default and Exposure at Default, etc. Will Pillar III ultimately be the undoing of VAR?

For example, if a large bank publishes a VAR of say 1% of total investments in a given period, but then takes a loss of 3% on those same investments in the subsequent period, investors (not to mention auditors and regulators) will discern pretty quickly that the bank's management is flying blind.

As former Fed Vice Chairman Roger Fergusson told Congress in 2003: "Pillar III--disclosure--will highlight any significant differences across banks, in the expectation that counterparties will penalize inconsistent risk measures."

In the event, regulators will be forced to recommend higher capital levels for those banks which are not good at predicting future losses, one reason why the Economic Capital simulation in the IRA Bank Monitor includes such prudent assumptions about future expected loss.

A number of organizations try to address the inadequacies of VAR by including obligor-specific factors in their risk models. Citigroup (C), for example, explicitly combines VAR methodology with factors that track the specific issuer risk in debt and equity securities. But far too many organizations simply accept the basic VAR result as good enough.

If as, as we suspect, the cost of risk is rising back to the mean after almost a decade of below-average experience, then the Federal Reserve Board unwittingly may be creating the circumstances for finally discrediting the use of VAR models for estimating specific financial risks. Consider the irony: Just as global regulators are enshrining VAR as the centerpiece of the Basel II regime, markets may demonstrate that this method of measuring risk is entirely useless -- at least when expected losses are not hovering around zero.

Over the past decade, we suspect that VAR models have appeared effective because there was so little risk to measure. But in an environment where risk events are large and "unusually" frequent, we suspect that financial institutions will quickly be forced to look for alternatives. Or in political terms, when the next unexpected event (or series of events) takes down a large financial institution, we expect to see trial lawyers and members of Congress quizzing the bankers, their familiars at the rating agencies and regulators on the efficacy of VAR for predicting specific financial mishaps.

During the inevitable process of recrimination and accusation that will occur following the next big systemic financial event, we hope that somebody asks members of the Federal Reserve Board and other federal regulators why they ever agreed to adopt into regulation VAR as a means of measuring bank capital adequacy. As we wrote in our comments on Basel II: "By relying upon the false assumption that financial market events tend to follow a random, normally distributed pattern, the FDIC and other regulators are about to adopt into [the Basel II] regulation one of the most specious, dangerous and widely held misconceptions in the financial world."

Comments

[Corporate G...:]

While VaR undoubtedly has its limitiations, this author is focusing purely on simple Gaussian parametric VaR which looks at historical offsetting correlations among asset returns and assumes normality of distributions. Normailty only tends to hold for delta-1 products with linear payoff functions such as cash equities, CFDs, swaps, futures, etc. When it comes to products with non-linear payoff fucntions and consequently non-normal return distributions such as options, structured products (ABS, CLNs, CDOs, etc.), prepayment on loans, contingent liabilities, etc., pararmetric VaR will grossly underesimate risk. Second, I would hope most practitioners are keenly aware that VaR does not capture liquidity risk. It simply captures market risk factors such as fx, equity, vol, ir, etc. Some models also incorpate credit VaR. For liquidity risk, you would need to calculate a completely set of different measures.

Therefore, it is no wonder why when markets fully correlate in downturns and liquidity dries up that your VaR figure will not work. That is why there is always the caveat that VaR only works in "normal market conditions". For this reason, this is is why we should not only use VaR, or believe everything the VaR number has to say. We should look at stress testing and scenario analysis as well.

To use VaR for risk capital allocation is admittedly dangerous, and with levarage and easy money these models were abused when capital was allocated over the past decade. There should be a buffer on top of the VaR figure. For instance, if VaR (usning historical Monte Carlo with volatilty scaling and decay factors ajdusted to represent the current market more accurately) as a % of risky assets is 10% then maybe we should hold a buffer of reserve capital twice that (that was an aribtrary suggestion but you get the point).

I am interested to see the VaR back-testing results the author refers to in Pillar III. In this way, we will see if the VaR models held up and were robust enough. Although in this market it is easy to say they didn't. But that is not necessarily because VaR is at fault. It is because banks either misused or minunderstood how to implement VaR and viewed VaR as the de facto measure of risk instead of using it as a means to get a rough idea about the possible risks being undertaken. The confidence levels of 99% 95% are misleading and seem to suggest that VaR is quite certain of the risks. Well, VaR is certain of certain kinds of risks, just not good at measuring leverage and liquidity, which is what bit us all in the ass in the end.

In closing, VaR should not be consirdered an absolute measure of risk but simply be used in relative terms to compare across banks assuming if banks ever implement consistent and comparable VaR frameworks, which is unlikely. Further, for risk capital allocation, we should put a buffer on top of what VaR tells us and not use the VaR number itself as a direction indication of how much cash to put aside or how much risk to take.

2008 Oct 24 01:48 AM

[jaded risk ...:]

I'm a quant. Bottom line is idiot management use models to make and justify risks of a business and not challenge them. Challenging assumptions is what models should be used for in risk management.

It's not simply they don't understand the risk - they do. Risk management has become a rubber stamp process at most firms, you don't get paid if you don't. Classic case of agency costs. Can't help it, the call-option bonus is how people are paid on the street.

Nov 13 01:34 AM