Anthony FJ Garner's  Instablog

Anthony FJ Garner
Send Message
Anthony Garner is a British national based in London. He left investment banking in 1992 in favour of a long cherished aim to work for himself and since 1995 has been trading financial markets for his own account, as well as having established, run or acted as consultant to a number of hedge... More
My company:
Malplaquet LLC
My blog:
Anthony FJ Garner
My book:
A Practical Guide to ETF Trading Systems
  • Back Testing - Tools For Fools? 0 comments
    Feb 27, 2013 2:57 PM

    At the end of the day, my belief, and certainly my experience is that however carefully I think things out, however much I back test, I am still constantly surprised at how in practice things turn out completely different. Well, perhaps "used to be surprised" is more accurate.

    Everything about back testing is a bit of a con really but as I have so often pointed out, it is all we have got to go on. A nice smooth equity curve results from many tests over many parameters of the same system on the same portfolio over the same time period. Well we all know what to think about that. Different time periods, different start dates, different portfolios must be looked at. Need I go on.

    But say you do not go on - you spend a long time on one system and convince yourself the level of drawdown, volatility and return all look realistic no matter how many time frames and parameters you test it over. Monte carlo analysis with re-sampling may help to bring you back down to earth. It tells you there is an alternate universe where those nice month long periods of increasing equity could easily be replaced by nice long periods of decreasing equity. It shows just how your equity curve MIGHT look or have looked if things had turned out differently. Random entry tests can also help you to see reality.

    And, funny old thing, time and time again I have noticed how the figures produced overall from countless monte carlo runs seem to bear a much closer relationship to the real life long term track records of the long surviving CTAs than any of my carefully planned theoretical equity curves do.

    Tools for fools - all of this stuff. At the end of the day we just have to stick our fingers in the air, see the direction of the wind and dip our trading toes firmly in the water, knowing that things are highly unlikely to turn out as we had expected.

    Just to re-iterate: I am not for a minute suggesting trading blind. It would be foolish indeed not to back test very thoroughly. But don't imagine for an instant that back testing has any predictive power. You will always face surprises and disappointments. As in every area of life you just have to do your best and accept that life will probably not turn out as you had planned.

    I do, very firmly believe in back testing and I am devoted to systematic trading. My real purpose was to point out how, if you are not very careful, you can so easily fool yourself.

    In the trend following world many point to the long track records of Dunn etc but fail to mention the countless thousands of CTAs which have bitten the dust for one reason or another. Let alone the psychologically un-livable with drawdowns - one of Dunn's programs is still in a 71% DD which started in the mid 2000s. I do believe in general though that long term trend following has a basis in reality, whether it is following broad long term economic growth through the stock markets or the advance / decline of a particular country/currency through the forex markets. Or whatever.

    I believe that simple, "cockroach" like systems can follow trends and that back testing does help to show that this has been the case in the past. Although as to the future....who knows.

    Curve fitting and fooling ones self are the real danger of back testing as well as unrealistic assumptions on a myriad of matters including the frictional costs of trading.

    The greatest unknown and danger in trend following is whether your strategy can survive the noise and the chop so that it can eventually benefit from the re-emergence of "clean" long term trends. Systems have to constantly adapt - hence the old, original Turtle methodology has bitten the dust - as least on the parameters and way they operated it. Tough game.

Back To Anthony FJ Garner's Instablog HomePage »

Instablogs are blogs which are instantly set up and networked within the Seeking Alpha community. Instablog posts are not selected, edited or screened by Seeking Alpha editors, in contrast to contributors' articles.

Comments (0)
Track new comments
Be the first to comment
Full index of posts »
Latest Followers


More »
Instablogs are Seeking Alpha's free blogging platform customized for finance, with instant set up and exposure to millions of readers interested in the financial markets. Publish your own instablog in minutes.