Jim Richmond

Can You Trust Monte Carlo Models?

Geoff,

The paper you recommended, like the other papers you've published on your site, was a great read. Thanks for the link.

I'm still not with you that long term MCS models need "ticker" level portfolio parameterization, but now I understand what you're saying. I still think you're blurring the portfolio management problem with the retirement planning problem but reasonable men can differ.

On a related note, I definitely misunderstood you on the inclusion of macroeconomic factors into the model. I guess at the ticker level that would get messy very fast. I thought when you mentioned the need for including forward looking predictive inputs that you were talking about macroeconomic parameters. I now see that you mean predicting the future returns/std dev at the ticker level based on fundamental analysis.

Again, since this site is called seeking alpha I need to tread carefully, but my focus (for portfolio mgmt) has always been at the asset class level, rather than at the ticker level. That's a debate we probably shouldn't get into because I know that's like religion. I do think there's alpha to be had at the security level, I just don't think I can easily or reliably find it or pay someone to.

I'm of the school that the most likely place to find sustainable alpha is by including macroeconomic factors as inputs, along with historical risk/return/correlatio... data, and optimizing at the asset class level. Now I've never built such a model myself, but I've used several and the approach seems sensible to me. I follow the work by the folks at indexinvestor.com and I think they've done some good work here.

Anyhow, interesting discussion. Thanks again,

Jim

Can You Trust Monte Carlo Models?

Geoff,

Sorry for leaving this conversation be for so long, it's actually really interesting to me.

So as we stand, we both agree that building Monte Carlo simulations that combine historic results with predictive macroeconomic models should be the current best practice for portfolio management and asset allocation decision-making. We both understand that these models are imperfect, but as of now, they are the state-of-the-art.

We also agree that if your MC simulation has to go out 30-50 years, you're signal to noise ratio (and the predictive value of the model) goes down considerably.

Where we differ (I think) is on whether it's necessary to incorporate cross-correlations between asset classes and predictions about macroeconomic conditions into long-term Monte Carlo simulation models that are used for planning purposes.

You believe that we have to at least try (since we had to do all the work for portfolio management purposes anyway), and I believe that it doesn't matter. Further, because of the complexity and costs it introduces (mostly measured by increases in simulation run-time), I think it's better not to.

Did I sum things up correctly? I'm not trying to convince you I'm right, but rather I'm trying to make sure we understand each other's position and justifications.

Regards,

Jim

Can You Trust Monte Carlo Models?

I think we may be violently agreeing, at least in part, especially with respect to putting too much stock on the historical record.

Where we perhaps differ is in our thoughts on our ability (definitely mine, maybe yours too) to build good models that predict future returns and standard deviations 50 YEARS OUT. I'm not saying we shouldn't try (that's why the site is called seeking alpha!), but for the average retiree, and maybe average planner in the mass market, it would be good to focus the portfolio survivability debate more accumulating enough wealth and on active withdrawal management techniques rather than getting too picky about modeling future returns for 30-50 years.

My point was that although a simulator that just takes a set of canned return and standard deviation pairs is rather trivial, it's a good enough place to start so you can focus more on the other variables that are much more under control of retirees such as savings and spending.

BTW, I'd gladly take my return/standard deviation inputs from a sophisticated model such as yours :) The point is that it's not where my energy is focused when working on the long horizon retirement planning problem.

That doesn't mean I don't think there's a place for advanced optimizers that combine past results with predictions about future macroeconomic dynamics. To me that's really cool stuff (way better than historical based models). I just think those tools are better suited to a 1-7 year horizon rather than a 50 year horizon. These tools are for portfolio management rather than for retirement planning. Does that distinction make any sense or is it a false one? I think the goals of the two exercises are different.

BTW, I got the thrust of Bernstein's argument on Monte Carlo to be that we shouldn't sweat the last 10 or 20 points of survival probability (80-85% is probably good enough) when interpreting Monte Carlo data. I thought his point was that because of fat tails from things like a future Hitler or the abomb, there's always a high degree of out-of-model variability that makes saying "we think you have a 95% probability of reaching your goal" a rather silly thing to say with any authority. Maybe I misread him.

In any case, thanks for the reply. It was good food for thought...

Can You Trust Monte Carlo Models?

This was an interesting post and I agree with the general comments on the variations of simulators being mostly dependent on differences in assumptions that are pretty understandable.

I must say however, that I disagree with the assertion that a simulator is useless unless it can generate average return and standard deviation from a real underlying portfolio.

IMHO, a major problem that financial planning (and perhaps finance in general) faces today is that we've built a beautiful edifice of solid mathematics on the very weak foundation that is our understanding of the true relationships between cause and effect in the return generating process.

While I hear your concern that the average investor would need a PhD in order to run many of the simulators, I don't think that obscuring all the built-in assumptions is necessarily better, especially if they turn out to be wrong!

It seems to me that in many cases, pulling out a return and standard deviation based on investor temperment and the historical record is not a bad place to start. I think some of these models have more error signal than they do data signal in them. The comment that the long term correlations go "all the way back to 2001" is at the heart of the challenge. And I'm not making fun of this. As I said, if you go back too far the error signal is louder than the data.

I've built an experimental Monte Carlo simulator that's focused more on the retiree's withdrawal methodology (ability to adjust the draw based on performance) rather than on guessing at the long term dynamics of each investor's underlying portfolio.

Based on my initial results (and as shown in other research), I'm beginning to think that working with retirees on managing their withdrawals and spending (as an ongoing process) may be more important (or at least as important) than trying to nail the precise details of the underlying dynamics of the portfolio and project them out 40 or 50 years into the future.

FYI, the simulator was written in Java and is available online at
home.comcast.net/~jsri...