Predicting Risk and Return for the QQQQ

One of the frequently asked questions about using Monte Carlo tools for portfolio planning is simply how to determine that the projected average return and risk levels for assets are meaningful. If you are interested in Monte Carlo and you have not asked this question, you should. At my firm, Quantext, we perform a wide array of validation studies to see if the output from our tools is valid. To address such a question, however, you must ask what a ‘valid’ result would look like. In this article, I show a series of validation tests for Monte Carlo simulations of the NASDAQ 100. The results have significant implications for investors in the QQQQ (QQQQ) ETF.

One of the basic tests of validity in a probabilistic investing model (i.e. a model like a Monte Carlo model that predicts a wide range of outcomes) is whether the projected uncertainty or risk levels are reasonable. In the world of professional risk management, this type of test is usually accomplished by comparing model output to the implied volatility in options markets. I have shown this type of validations for the Quantext Portfolio Planner (QPP) and Quantext Retirement Planner (QRP) in a range of articles—most recently here (pdf.). A layman’s introduction to the basic process of validating projected volatility by looking at options markets is provided here. Note that QRP and QPP use the same basic analytics, so I will simply refer to QPP from here on out.

Along with validating the projected future risk (i.e. volatility) for an asset, we also want to know that the projected average return from the model is meaningful. Once again, we have to think about the basis for determining validity. I tend to compare the predicted results for three years into the future from our Monte Carlo tools to the trailing performance over the past three years. If, for example, QPP can consistently predict the average return on an asset over the next three years more accurately than simply looking at the trailing three-year average return and do this consistently, I consider the QPP projections to be meaningful. There is nothing special about the three year time horizon except that I tend to favor the use of the last three years of market data in QPP—a user defined variable in QPP, but the default setting is three years.

In the sections below, I show a series of results generated using the default inputs for QPP and generating predicted average annual returns and standard deviation in annual return for the NASDAQ 100 index.

Historical data from 1973-1975 is fed into QPP as the basis for predicting the average return and volatility in return for the NASDAQ 100 from 1976-1978, etc. Going through the end of 2005 gives us ten distinct three year periods with no overlap.

The results of the simulations are shown above. The most relevant total statistic for looking at the quality of the projections is the average error of the prediction for any period—whether positive or negative. This measure is called the Mean Absolute Error [MAE]. If QPP predicts an average annual return of 10% and the actual average annual return is 8%, the MAE is 2%. If QPP predicts an average annual return of 8% and the actual average return is 10%, the MAE is also 2%.

This is important because if you just looked at average error and you have one period in which QPP predicts 10% too high and one period in which QPP predicts 10% too low, the average error would be 0%. The MAE in this case is would be 10%. The MAE for the QPP-projected average annual return is 11.5% over these ten three-year periods.

Is that good or bad? This is where having some benchmark comes in. If we think of using the trailing three-year average annual return as an estimate of expected future return on the NASDAQ 100, the MAE in annual average return is 21.1%. This means that using the results from the most recent three years as your expected return for the next three years will result in almost twice the error in your outlook than using QPP’s projected average annual return.

How significant is a projected average annual return MAE of 11.5%? If we calculate the calendar year returns for each year in the 30-year period, the average annual return for the NASDAQ 100 is 14.78% per year. If we had magically known this value ahead of time and used this single value as our estimate of the average annual return in every three year period, the MAE of our estimates would be 11.13% per year. QPP, in its projections, used only data available prior to the beginning of each forecasted period and has an average error [MAE] in projected average annual return that is only 0.37% worse (11.5% vs. 11.13%) than you could have made if you magically knew the future average annual return for the NASDAQ 100 over the 30-year period ahead of time.

The QPP projections for the standard deviation in annual return are also considerably better estimators of the future volatility of the NASDAQ 100 than using trailing data alone. The MAE for predicting the standard deviation in annual return using the trailing three-year value is 8.4% while the MAE using QPP is 6.6%. This result is not surprising since we have shown that QPP’s projected volatility tends to match implied volatility and implied volatility is known to be an unbiased estimator for future realized volatility (as discussed in the two papers cited in the second paragraph of this article.

At this point, I have shown that QPP’s projections (using only data available prior to the target period) for average annual return for ten three-year periods from 1976-2005 for the NASDAQ 100 are far better than using the most recent three years and almost as good as if you had had an omniscient guide who told you the average annual return for the NASDAQ 100 for the 30-year period ahead of time. I have also shown that the QPP-projected standard deviation in annual return is a better forward-looking estimate than using the trailing three years.

If you have a more accurate estimate of average return for an asset (say QQQQ) and a more accurate estimate of the standard deviation in the returns on that asset, you can make better decisions regarding asset allocation. Many investors and advisors place too much weight on recent performance of an asset class (risk and return). QPP and QRP will tend to be contrarian in such circumstances tend to support asset allocations that are more consistent with the long-term balance of risk and return across asset classes.

Using the three years up until the end of 2005, QPP projected an average annual return for the NASDAQ 100 (i.e. QQQQ) of 11.4% with a standard deviation of 21.4%. Over the three years to the end of 2005, QPP reports a trailing average annual return of 18.72% over the previous three years, with a standard deviation in annual return of 14.6%. Which set of statistics will be a better estimate of the future? I am willing to bet that the QPP projection is a more reliable basis.

While I have shown the results for the NASDAQ 100, I have also performed the same tests for a variety of indices and portfolios, with similar results. This type of testing helps to demonstrate that a good Monte Carlo model can provide estimates of future risk and return that will be helpful in portfolio planning and asset allocation—considerably more helpful than looking at trailing history. If you approach asset allocation using results over the recent 3-5 years, your portfolio will tend to follow rather than lead trends and the costs are high. William Bernstein has shown this very clearly in The Intelligent Asset Allocator, as have many other studies. Bernstein looked at allocating portfolios using the trailing five years of historical data into six major asset classes for the period from 1975-1998. He then looked at performance in each subsequent five year period. He found that the portfolio that is optimized to the trailing five years generated an annualized return of 8.4% per year while a simple allocation equally into all six asset classes generated 15.79% per year. This study is well worth heeding.

Since you cannot make decisions using the straight historical data, I believe that the best approach to estimating the future risk and return of your portfolio is to use a forward-looking Monte Carlo solution like QPP. A Monte Carlo model like QPP succeeds not by predicting the future so much as simply assuming that the long-term balance between risk and return among asset classes and for asset classes through time will be maintained. When you examine asset allocations using this type of approach, the most striking thing about the projected future results is that the asset classes that have under-performed relative to their risk levels are projected to do better and those that have out-performed relative to risk levels are projected to return less. These results ultimately look a lot like common sense, with the advantage to the user that the QPP results provide a consistent estimate of the risk-adjusted return that you can reasonably expect across assets and accounts for the cross-correlations between assets.

In future articles, I will show the same types of tests for other asset classes and for various portfolio allocations.