Why a System is Necessary
What is a model (or trading system) and why is it necessary? A trader (or investor -- the argument applies equally to both) must have confidence and discipline. The trader without a system will inevitably succumb to poor decisions. If the model generates a series of bad calls, the trader decides that it does not work. Even worse, a trader may use the system as a list of "suggestions" and only follow those that appeal.
What has happened in these cases is that the trader has substituted his own methods for those of the model. That might also work, but only if the trader has a good method. The model or system is no longer relevant, since the trader is driving the bus. Darrell Jobman writes a nice description of how this can happen in the current SFO Magazine (free registration required).
Developing a Model
There are a number of steps involved in developing a successful trading system or model. This is the short course. As readers will see, there are many traps.
• Assemble a good development team. If you our your partners do not understand how to develop a system or model you will fail -- most likely by developing a backfitted model.
• Start with theory, including some ideas about technical analysis and how markets work. (If one just begins with data and looks for relationships, the result is backfitting).
• Find robust measures of key variables. Do not "optimize" parameters to find the best possible value. Look for values that work well even if they are changed higher or lower by a reasonable amount. (If one does not do this, the result is backfitting).
• Do not use too many variables. Get rid of as many as possible, even if your theoretical performance declines. (Using a lot of variables results in backfitting).
• Make sure you have plenty of data. Trading systems require dozens (at least) of instances that you are trying to predict. Hundreds of relevant cases provides much more confidence. This means that you need many years of data to get plenty of cases. (One must not use all of the data, however, or you will have a model that has perfect backfitting).
• Do out-of-sample testing from different eras with different market conditions. (Do not keep using the same out-of-sample data for dozens of model tests, however or you will develop backfitting).
• Look carefully at the simulated results and equity curve. Pay close attention to the streaks of wins and losses. This is the reality with which you must live when you trade the model. If these streaks and drawdowns are unacceptable, the model is unacceptable. Start over.
The result of this process is a model that provides reasonable edge with reasonable risk. If it seems too good to be true, you have probably made a mistake. Most likely, you have done something that resulted in a "peek forward" or have used so much data that you have backfitting. But by now, the astute readers of "A Dash" have figured that out!
One of Our Models
We are going to illustrate this process with one of our models: TCA (trend, cycle, anticipation). We do not present this as a trading recommendation -- please see our disclaimer-- so we are not showing every signal and the reports are delayed. The purpose is to show traders and individual investors what they will experience even with the very best professional models. We confidently predict that the model will make some calls that are losers -- about 45%. We know this from the simulation data.
The brief description of the model is that stocks and indices have periods of trending and periods of cyclical behavior. Vince has attempted to identify both and find entry points. He has also built in a factor for anticipation. This leads both to faster entries and faster exits. This is not a simple "mo-mo" model. The average holding period is about one month, but it can be much shorter if a signal "fails."
We tricked Vince a little on this one. We took a model that he developed using the procedures described above. We retained an outside consultant who reviewed the model on data not used by Vince. The expert gave glowing reviews. Finally, we backtested it on something that Vince had not even considered, a portfolio of iShares. We chose iShares because of the great liquidity and scalability, the ability to develop a diverse portfolio, and the potential for great diversity in trading choices. Pleased with the results we found, we are now using it in our funds.
Readers should note well that this is an educational process, not a trading recommendation. While we shall report results, we are not making recommendations or giving every signal in real time.
Learning from the Model
After many years of following trading models we have learned an interesting method for following them. We pretend that TCA is a wise and successful technical analyst (since the model is based on technical factors). TCA makes a call and we follow it. We look at the chart to learn something about what the model is telling us. We invite comments from technical experts like Dr. Brett, Trader Mike, Maoxian, Charles Kirk, Adam Warner, Bill Rempel, the inimitable Muckdog, and others whom we read daily.
We understand that no system is perfect. Far from it. It takes only a small edge to make significant returns, and few can accomplish this. It is our confidence in the development process that allows us to ride out the false signals, expecting a strong risk-adjusted return. Readers of "A Dash" will see the similarity to the lesson from the Blackjack Players.
This week we had a signal to buy iShares S&P GSTI Software Index (NYSEARCA:IGV), a software index including leading holdings of Microsoft (NASDAQ:MSFT), Oracle Corp. (NYSE:ORCL), Adobe Systems (NASDAQ:ADBE), Symantec Corp (NASDAQ:SYMC), Electronic Arts, Inc. (ERTS), Computer Associates (NASDAQ:CA), Autodesk (NASDAQ:ADSK), Intuit, Inc. (NASDAQ:INTU), Citrix Systems, Inc. (NASDAQ:CTXS), and Cadence Design Systems, Inc. (NASDAQ:CDNS).
We shall follow up on this signal in future articles, and also present additional examples.
We will also revisit the various mistakes that model developers might make, showing the possible pitfalls in greater detail.