How Data Mining And Arbitrage Impact Smart Beta

by: Brian Haskin

Smart beta strategies weight their investment exposures to emphasize risk factors that have been "anomalous" in the past - that is, they've outperformed market cap-weighted indices. These anomalous risk factors include value, momentum, low volatility, small size, and many others that have been "discovered" through historical backtesting. But how much of this historical outperformance has been cherry picked from the data? And what impact, if any, has discovery of new factors had on their persistence?

Impact of Data Mining

According to a new white paper from Evanston Capital Management, smart beta strategies have seen their returns fall between 30% and 71% short of expectations in "out of sample" periods - i.e., before and after the scope of backtests. This suggests that data mining (i.e. "cherry picking") and arbitrage (i.e. exploiting discovered "anomalies" to the point they begin to disappear) play a "material, economically significant role in performance expectations."

"Smart beta is one of the most popular, cutting-edge investment products available today," said Evanston Senior Investment Strategist Peter Hecht, Ph.D., in a March 21 press release. "As is the case with many investment products, the largest risk confronting smart beta investors is what to assume about returns on a prospective basis."

Dr. Hect and Ph.D. candidate Zhenduo Du are the authors of Evanston's white paper, which is titled Smart Beta, Alternative Beta, Exotic Beta, Risk Factor, Style Premia, and Risk Premia Investing: Data Mining, Arbitraged Away, Or Here To Stay? According to Adam Blitz, CEO and CIO of Evanston Capital Management, the paper's goal is to "educate both investors and financial advisors" on how to "better navigate the evolving environment." Its key findings include:

  • Backtest data influence investor expectations of how smart beta strategies will perform in the future, but data mining taints many of the backtests;
  • As factors move from proprietary or private to being widely known, at least a portion of the factor's previous outperformance will be arbitraged away;
  • The combination of tainted backtesting and arbitrage have resulted in "out of sample" performance that greatly lags "in sample" performance: by as much as 71% by one measure, and 58% when correcting the backtested realized average returns.

"Unlike other products, smart beta strategies have a few key features that make predicting returns on a prospective basis interesting, yet potentially problematic," said Dr. Hect. "While historical backtests are indeed helpful, we found that without fully appreciating the impact of data mining and arbitrage, the over-reliance on historical backtesting can create a false sense of confidence about future return performance."

For more information, download a pdf copy of the white paper (linked above).

Past performance does not necessarily predict future results.
Jason Seagraves contributed to this article.