Entering text into the input field will update the search result below

The Perils Of Backtesting Technical Strategies

Feb. 14, 2012 7:59 AM ETRSP, SPY1 Comment
Jeff Gonion profile picture
Jeff Gonion

Backtesting is the process of evaluating a strategy using historical data as input, usually by measuring the performance of the strategy over time. It is a popular technique used by both individual and professional investors alike, but should be used with caution.

For example, growing weary of your net worth evaporating during recessions, you might decide to backtest owning the S&P 500 only when it's above it's 200-day moving average, thus avoiding the big recessions. You could test this theory on historical data for SPY to see how it performs. Backtesting this strategy reveals that it avoids recessions as planned, but also locks-in many smaller losses, while avoiding the rebounds that typically accompany them. Backtesting helps you realize that perhaps this isn't such a good idea after all.

When backtesting a timing strategy on a single equity or index, care must be taken to ensure the backtest does not inadvertently use future or coincident data that would not be available to base decisions on. Otherwise, backtesting simple timing strategies is fairly straightforward.

Backtesting stock-picking strategies is entirely different, fraught with subtle complexities that can easily lead to incorrect conclusions. In order to correctly backtest stock-picking strategies, a complete historical database of the entire stock market is needed, including companies that have either gone bankrupt or merged with other companies.

A common technique among individual investors is to backtest strategies using historical data for stocks that currently exist, which ignores companies that have merged or gone bankrupt. This causes "survivor bias" in the backtest, and results in backtests that are overly-optimistic.

The further back in time the test goes, the greater the number of bankrupt companies that will be ignored, and the greater the survivor bias, especially if the results are compared to real-world stock indexes which do not have survivor bias. This is also

This article was written by

Jeff Gonion profile picture
Using economic and computational/statistical models to guide investment decisions

Recommended For You

Comments (1)

Can backtesting actually be used to develop viable strategy that works in the real market?

Yes it can, but only in the hands of very skilled people. Renaissance Technologies and James Simons come in mind, and there are
others of course, who use complex mathematical models to analyze and execute trades in the markets with spectacular long term results.
Disagree with this article? Submit your own. To report a factual error in this article, . Your feedback matters to us!
To ensure this doesn’t happen in the future, please enable Javascript and cookies in your browser.
Is this happening to you frequently? Please report it on our feedback forum.
If you have an ad-blocker enabled you may be blocked from proceeding. Please disable your ad-blocker and refresh.