Accurately Measuring Mutual Fund Risk

by: AlphaBetaWorks

Summary

Risk management and selection of skilled fund managers both rely on the ability to measure exposures to common risk factors, such as market risk, accurately.

Techniques for measuring market risk can be complex and their effectiveness hard to assess.

In this piece we evaluate the effectiveness of several techniques for measuring market risk of U.S. equity mutual funds.

To evaluate the accuracy of market risk (beta) estimates, we test their hedging effectiveness. Specifically, we evaluate realized market correlations of hedged U.S. equity mutual fund portfolios.

Whereas risk estimates that assume a fixed 1 beta and estimates that use returns-based style analysis (RBSA) are flawed, a statistical equity risk model applied to portfolio holdings is close to ideal.

Market Exposure (Beta) Estimation Techniques

We analyze approximately 3,000 non-index U.S. Equity Mutual Funds. These provide a broad sample of the real-world long equity portfolios that investors may attempt to hedge. We evaluate the effectiveness of three techniques for calculating market exposures (betas):

  • Assuming constant 100% market exposure (1 beta): Absent deeper statistical analysis, it is common to assume that all portfolios have the same market risk, equal to that of the broad benchmarks.
  • Using returns-based style analysis (RBSA): RBSA is a popular technique that attempts to estimate portfolio factor exposures by regressing portfolio returns against factor returns.
  • Applying a statistical equity risk model to portfolio holdings: This technique essentially performs RBSA on the individual portfolio holdings and aggregates the results.

For each fund and for each month of history we calculate market exposure at the end of the month and then use this estimated (ex-ante) exposure to hedge the fund during the following month. We then analyze realized (ex-post) 12-month hedged portfolio returns and calculate their correlations to the market. The lower this correlation, the more accurate the technique and the more effective it is at evaluating the risk of a typical U.S. equity mutual fund portfolio.

Perfect Estimates of Market Exposure and Perfect Hedging

A perfect estimate of Market exposure should produce perfect hedges and zero mean and median market correlations of hedged portfolio returns. Yet, if sufficiently large, even a set of perfectly random 12-month return series will contain some with large market correlations. Since our study covers over 200,000 12-month samples (observations), some market correlations are close to 1 by mere chance. To account for this and to create a baseline for comparisons, we calculated market correlations for random return series with a Monte Carlo simulation. These results, attainable only with a perfect hedge, are the baseline against which we evaluate equity market exposure estimation techniques:

Realized 12-month market correlations of random return series

Source: abwinsights.com

Min. 1st Qu. Median Mean 3rd Qu. Max.
-0.9243 -0.2162 0.0001 -0.0006 0.2146 0.9409

Estimates of Constant Market Exposure

A naive assumption that market exposures of all portfolios are 100% (market betas of all portfolios are 1) is obviously wrong:

U.S. Equity Mutual Funds: Realized 12-month market correlations of portfolios hedged using a 100% market short

Source: abwinsights.com

Min. 1st Qu. Median Mean 3rd Qu. Max.
-0.9956 -0.2927 0.0469 0.0148 0.3527 0.9958

This approach over-estimates the risk of some portfolios and under-estimates the risk of others. There is a group of low-exposure portfolios for which a fixed 100% market short is too large. These produce a fat tail of negative market correlations of nearly -1 for some hedged portfolios. There is also a group of portfolios for which a fixed 100% market hedge is too small.

Similarly, any performance evaluation and manager selection system that blindly evaluates returns against a fixed benchmark with no regard to portfolio risk will be flawed.

Estimates of Market Exposure using Returns-Based Analysis

Returns-based style analysis with multiple factors suffers from known issues of overfitting and collinearity. Less well known are the problems that arise from RBSA's assumption that exposures are constant over the regression window. In practice, portfolio exposures vary over time and can change rapidly as positions change. RBSA will capture these changes months or even years later once they influence portfolio returns, if at all.

RBSA thus fails similarly to the fixed exposure above, if less dramatically: risk estimates are too large in some cases and too small in others. The exposure estimates are biased, since they produce hedges that are too large and market correlations that are negative, on average:

U.S. Equity Mutual Funds: Realized 12-month market correlations of portfolios hedged using returns-based style analysis

Source: abwinsights.com

Min. 1st Qu. Median Mean 3rd Qu. Max.
-0.9946 -0.3229 -0.0518 -0.0459 0.2274 0.9762

Consequently, any performance evaluation and manager selection system that relies on returns-based analysis of returns will be flawed when portfolio positions and factor exposures vary over the regression window.

Estimates of Market Exposure using a Statistical Equity Risk Model

The Alpha Beta Works Statistical Equity Risk Model produces risk estimates close to the ideal:

U.S. Equity Mutual Funds: Realized 12-month market correlations of portfolios hedged using a statistical equity risk model applied to holdings

Source: abwinsights.com

Min. 1st Qu. Median Mean 3rd Qu. Max.
-0.9565 -0.2230 0.0184 0.0214 0.2669 0.9755

The model estimates security market exposures using robust regression methods to control for outliers. Though robust techniques perform well for most portfolios, they appear to produce hedge ratios that are too low for some high-beta portfolios. This leads to small positive mean and median market correlations of hedged portfolio returns and to the higher probability of positive market correlations compared to random portfolios.

Aside from this under-estimation of a small fraction of portfolios, application of the Alpha Beta Works Statistical Equity risk model to fund holdings comes closest to perfect equity market exposure estimation and close to perfect hedging. Portfolio managers and investors who rely on robust risk models and exposure estimation techniques can thus nearly perfectly hedge the market risk of a typical equity portfolio. Performance and skill analysis built on this foundation will be similarly effective.

Summary

  • All performance evaluation and manager selection, sometimes implicitly, relies on estimates of portfolio factor exposures. The most important of this is market exposure (beta).
  • The effectiveness of equity market exposure estimation techniques can be assessed by comparing hedged portfolio returns to random portfolio returns that would be produced by perfect exposure estimates and a perfect hedge.
  • Simplistic exposure estimates that assume 1 beta for all portfolios fail most spectacularly for low-risk portfolios. Analysis that uses a fixed benchmark for evaluating portfolios without regard to actual factor exposures will be flawed.
  • Returns-based style analysis (RBSA) both overestimates and underestimates market exposure, likely due to its failure to capture rapidly changing risk. Analysis that uses regressions of returns for evaluating portfolios will be flawed in cases where portfolios and exposures vary significantly over the regression window.
  • Analysis of fund holdings using a Statistical Equity Risk Model comes closest to perfect equity market exposure estimation. Performance evaluation and manager selection built on this foundation will be most predictive of the three approaches.

The information herein is not represented or warranted to be accurate, correct, complete or timely. Past performance is no guarantee of future results. Copyright © 2012-2016, AlphaBetaWorks, a division of Alpha Beta Analytics, LLC. All rights reserved. Content may not be republished without express written consent.

Disclosure: I/we have no positions in any stocks mentioned, and no plans to initiate any positions within the next 72 hours.

I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it. I have no business relationship with any company whose stock is mentioned in this article.

About this article:

Expand
Want to share your opinion on this article? Add a comment.
Disagree with this article? .
To report a factual error in this article, click here