Measure, Don't Model: The Forest and the Trees 25 comments
-
Font Size:
-
Print
- TweetThis
A word of warning … while the concepts in this post are important, I respond to a couple of commenters who used some mathematical terms. So to respond, I use those terms too, plus a few more, as well. Try not to get distracted by the jargon. Instead, pay attention to the concept, which is important.
In the interest of keeping my blog posts mainstream, I try to leave out the big words that are unnecessary. And sometimes, because of that, and because of space contraints, I omit factors that just don’t have a significant impact. Well, my last post generated a few comments, both here and at Seeking Alpha. I feel compelled to respond.
Dave correctly notes in the comment section that a normal distribution with fat tails is leptokurtotic (”LK”). While I am well aware of what it means, the word still makes my head hurt. That said, it’s pretty well known, at least in the quant world, that the market is somewhat LK. The thing is, when you read that Bloomberg article that started all of this, you’ll see that the range Bloomberg is citing is based on a perfect normal distribution that is not LK. The fact that Bloomberg would cite the probabilities that are derived from a perfect normal distribution is one issue.
But here’s something far more important. There are literally hundreds of different volatility models out there. Some of the more prominent models are Engle’s Nobel-winning ARCH model and its many derivatives (such as GARCH), to Heston’s Stochastic model, to jump-diffusion, to whatever. I read one research paper published in 2001 that evaluated 330 (not a typo!) different volatility models … 330!! And all of them were some form of derivation of the normal distribution. Of all the different models considered, none of them beat GARCH (1,1), which is based on a variance, which is based on a normal distribution, which is based on the assumption that each day’s price action is independent. And here’s what is significant about that: None of these models sufficiently accounts for the possibility of gigantic moves back-to-back-to-back.
Another recent research paper compared various models to Black-Scholes using S&P 500 data — precisely what we were looking at. In that research paper, they found that a “shape-adjusted” Black-Scholes beat everything. That shape adjustment follows a process proposed by Li and Pearson (2005). But again, the problem remains that all of these models consider each day independent. So a 10-standard deviation daily move, followed by another daily move that size, followed by another, is considered practically impossible. But that is what happened in the credit markets last year, which is why a skewed, leptokurtotic shape adjustment is insufficient.
A different issue, which was discussed in the comment section over at Seeking Alpha, was the fact that I didn’t include “drift” or “mean expected return” as a factor. Well, let’s keep it simple: 12 years ago, the S&P 500 was at 900. Now, it’s at 900. What’s the return over the past 12 years? Zero! Want to look at the returns by month? Let’s do that, and add a layer to the complexity and factor in dividends and then calculate the mean monthly return over that time frame. You get 0.0019.
Here’s my question. Why add complexity to the process and try to account for mean expected return, when that return is so close to zero? It’s as if I should also factor in an interest-rate-discounted stock price (something that OptionVue does do). Perhaps that might be important if rates were double digit, and the duration was a year. But LIBOR rates are below 1%, and the analysis period is only 30 days. That’s 0.0003. Again, it’s so close to zero that it’s ridiculous.
Listen, I don’t want to discourage comments. I really do appreciate all that is said. But these particular comments strike at the heart of the problem I see. The problem isn’t that the model-predicted range was off by 0.0019 because nobody failed to consider the mean expected return drift. And the problem isn’t that skew and kurtosis aren’t considered — as noted above, there are numerous models that consider both. Those are just “trees”. The problem was and still is that these types of models are based on independence. That is, they’re predicated on the concept that one action does not lead to another. As the action in 2008 proved, where one event (Lehman’s bankruptcy) led to another (Reserve’s Prime Money Fund breaking of the buck), which led to another (a “bank run” on money market funds), which led to another (a freeze in LIBOR and the commercial paper markets), etc., The failure to account for dependence created is what’s significant. That’s the “forest”.
Thanks for listening. Now, back to work.
Related Articles
|
-
The Seeking Alpha 100
See allThe top 100 stock
market authors
selected for publication
Seeking Alpha 100 » -
Fastest Climbers
Authors with the largestSee all
increase in followers
in the last 7 days
Fastest Climbers » -
Top Commenters
Top commenters,See all
as ranked by their peers
Top Commenters » -
Top Instabloggers
The 100 most active
Instabloggers.-
1
David Fry
-
2
TraderMark
-
3
Don Dion
-
4
Cliff Wachtel
-
5
Mike Havrilla
Top Instabloggers » -
1
-
Top StockTalkers
TopSee all
StockTalkers.
Top StockTalkers »
This article has 25 comments:
I think it is linked to the way that our brain stocks information in small increments and does not connect the dots unless you force it to.
Did I get the concept right? You are saying it is difficult to model any string of days even though the days themselves might conform to some model because the days act independently, and you can have bizarre actions in a single day regardless of the overall trend? In addition, you can have a string of bizarre days in a row.
- "12 years ago, the S&P 500 was at 900. Now, it’s at 900. What’s the return over the past 12 years? Zero!" The time period is cherry-picked to make a false assertion about expected returns. Here's a forest: would anyone invest in the stock market if the expected return was zero? There's an undeniable long-term upward trend to the market (roughly 7-8% annually, or .60-.65 monthly). To make the measurements you're making, or at least to make them accurately, it should be included. Measuring accurately may be concerned with the trees, but measuring inaccurately can put one in the wrong forest. It’s especially important when assigning a probability against which a subsequent comparison will be made.
- From the original article: “It (your tool) compares the probability assumed by volatility (which is calculated assuming a normal price distribution) to the actual probability of a certain sized stock movement.” You’re correct, return data doesn’t follow a normal distribution. This is widely accepted. Mandlebrot and Taleb have covered this topic, and its implications, extensively. They’ve also covered your observations about the connectedness of events and market memory. That these concepts are still widely ignored by mainstream commentators and experts is self-evident. You write “That means the index was stagnant far more frequently than predicted.” The other side of that coin is that it also delivers extremely large moves more frequently than predicted. But, both these observations are tautologies given that the predictions (taken from the VIX) are, as already noted, variance-based and wrong. “Actual” probabilities, based on history are no help, particularly for single stocks. The moves in financial issues in 2008 should convince anyone. Betting on stagnation will, at some point, deliver the painful lesson of picking up nickels in front of steamrollers. And, as you note, that’s a very serious mistake to make.
I certainly don't think markets are efficient, but expecting to be the market participant who identifies a pattern before everyone else, buys before everyone else, and sells before everyone else sounds like a fool's errand to me.
I ask that, in the future, you not begin by saying I'm falling into some trap. It really grates the nerves. And I've got work to do. With that said, I can't resist ...
I'm not about to say that you're wrong about drift. If you want to factor in 0.60% per month into the calculation. Do it ... No wait, I'll do it. It's a two-minute change to my program.
Done ... Factoring drift into the equation causes the percentages for the past 18 years go from 83.85% to 84.88%. If you think that's significant, so be it. My concern is where there are differences. Looking at the drift impact broken down by year, the only time there is a significant difference is ... well, I'll let you answer that question.
Now let's look at your second paragraph. I don't know where to begin. You begin by saying I'm correct about return data not following a normal distribution. Then you say my correct statement is widely accepted. I say you're wrong. It's not widely accepted. The fact that the Bloomberg article even exists shows it's not widely accepted. Yes, it's more accepted now than it was a few years ago. But widely accepted? Then explain to me how bell curve calculations end up on Bloomberg.
You then say the topic of complexity and connectedness is widely ignored. Fair enough, although that was the crux of Bookstaber's testimony to the Senate last year.
But finally, you quote me as saying "That means the index was stagnant far more frequently than predicted." But what you omit is the context. You leave out the next sentence, where I said, "the VIX was never accurate, at least when it came to 1 standard deviation". You leave out what I said two paragraphs later where I asked, "While the VIX may have improperly modeled 1 standard deviation, what about other standard deviation ranges?" So yes, in that one sentence, I did say the market was more stagnant than suggested. But in given the fact that the next sentence I said, "at least when it came to 1 standard deviation", I thought it was clear that I was talking about 1 standard deviation.
And as far as tautology -- nice choice of a word -- how is it unnecessarily redundant and "the other side of the coin" to say that the market stays within 1 standard deviation *more* than normal and to also say that there are far too many "3, 4 and 5 standard deviation moves than the bell curve suggested." More of each!! That's doesn't sound like the other side of the coin. And it's not unnecessarily redundant! You get more of both: more small moves and more extremely large moves than expected. That's counterintuitive. [The flip side is that the number of moves between 1 and 2 standard deviations is fewer than expected.]
In sum, I'm glad you agree with my major points. But lighten up on the "falling into some trap" crap. Worrying about drift over a 30-day period, and taking things out of context are the real traps.
On Jan 09 09:35 AM sumguytang's wife wrote:
> I'm a beginner in understanding the markets, and equally a beginner
> in complex mathematical concepts, let alone the terms used to describe
> those concepts. I almost abandoned this article after reading the
> caution at the beginning, but taking the admonition to tolerate the
> jargon and pay attention to the concept to heart, I slogged ahead.
>
>
> Did I get the concept right? You are saying it is difficult to model
> any string of days even though the days themselves might conform
> to some model because the days act independently, and you can have
> bizarre actions in a single day regardless of the overall trend?
> In addition, you can have a string of bizarre days in a row.
Easy fella. C'mon, you write that folks are not seeing forest for trees, but then get uptight when they write your falling into the same trap. What's good for the goose, pal. You cherry-picked data (a tree) and ignored the forest.
The rest of my comments stand. Please calm down and read them agian. The population goes beyond self-descirbed experts and self-promoted commentators. The ideas are out there and widely accepted.
Now, this may really set you off so let me apologize in advance. You haven't discovered anything new and you are making some serious errors in probabilistic thinking and the associate outcomes. That's the same thing you find fault in others for doing.
Couldn't have said it better.
I will note one thing -- at least in the academic community -- they don't like simple. That's because it doesn't impress anybody. The more elegant the equation, the more likely you'll get that PhD. So there is a huge disincentive among quants in academia to try to come up with a simple solution.
On Jan 09 11:19 AM Chris B wrote:
> The takeaway lesson is that quantitative strategies in the stock
> market are inherently self-defeating. Hundreds of thousands of very
> smart people with advanced math and IT training are analyzing gobs
> of market data every day, trying to find some pattern that will predict
> future results. When the data indicate a pattern, they will all
> plow in and obliterate whatever arbritage opportunity or variable
> relationship they have identified. This behavior changes the nature
> of the market that their model once predicted, making the model obsolete.
> That's why no one investing method always outperforms all others.
> If that were true, everybody would soon be following that method,
> bidding up prices and reducing returns in concentrated areas. Cyclical
> markets exist for a reason.
>
> I certainly don't think markets are efficient, but expecting to be
> the market participant who identifies a pattern before everyone else,
> buys before everyone else, and sells before everyone else sounds
> like a fool's errand to me.
In academia, the most elegant solution most often IS the simplest.
e = mc^2
is perhaps the most well known example.
On Jan 09 12:00 PM odds wrote:
>
> I will note one thing -- at least in the academic community -- they
> don't like simple. That's because it doesn't impress anybody. The
> more elegant the equation, the more likely you'll get that PhD.
> So there is a huge disincentive among quants in academia to try to
> come up with a simple solution.
Here's where you and I disagree. You say I cherry-picked. But then you don't identify where. So I'm left to guess that you're talking about my selection of the 10-year period where I said the drift was zero.
Okay, so I chose your numbers, and, as noted, the probabilities changed by ... 1%. So what? Here's where it's critical you have context. It's one thing to worry about 1% when you're risking $98 to make $2 (*). If the actual probability of winning goes from 99% to 98%, or from 98.5% to 97.5% that's a real problem. As risk and reward get even further imbalanced, probability errors get more significant. But when the trade is risking $67 to make $33 and the probability goes from 83 to 84? Worrying about 1% difference is insignificant mainly because models are imprecise in the first place.
And that's the crux of what I've tried to say, more important than anything else, readers need to know that these are models we're talking about. They have errors. They're not precise. They never have been. They are not reality!! Meanwhile, you're worried about drift over a 30-day period that takes probability from 83.8 to 84.8? Is that an edge that you're willing to trade off of? People that get hung up on that kind of precision and take the numbers as gospel are the ones that get hammered.
You're also inconsistent. In one sentence you agree that the bell curve is bunk, then you go off on me omitting drift from ... the bell curve!
You also say that what I'm talking about is "widely" accepted. We disagree. Maybe in the circles you work in. But I say the Bloomberg article and a wide array of other articles, plus all the option tools that still use Black-Scholes and other similar models, prove that, at least to the typical Seeking Alpha reader, it's not widely accepted.
You also say that I haven't discovered anything new. Did I claim to have done that? Like I said in my first article, I've been using different probability models since 1996 -- nothing new. And I never claimed they were! What is new is the FREE TOOL.
Which gets to the beef I have with your comments. If I didn't claim to be doing something new, why did you say what you said, knowing that "this may really set me off"? Why say, "You haven't discovered anything new", when I never said I had? Just trying to incite?
Finally, you claim I've made serious errors in probabilistic thinking, I heartily welcome a correction.
* - Not advocating that a trader risk 98 to make 2, unless it's via a T-bill. Just making a point. And for those who don't think that anyone risks 98 to make 2, lottery commissions, casinos and insurance companies risk 98 and more to make 2 all the time.
-- Don
But that calls to mind the problem with these statistical techniques: they were designed to analyze phenomena which are themselves unaffected by the historical predictive success of the algorithm: whether my face recognition algorithm works well in low light does _not_ change what faces look like.
The historical success of an algorithm used to understand financial markers does, however, affect the markets its describing. It becomes embedded in assumptions of risk and the distributions of returns, for example, and the existence of many positions built on a foundation of assuming those historical distributions of returns, in fact prospectively changes the distribution of expected returns (though the modelers don't know it).
This is part of the "dependence" of which you speak, and was ignored at great cost.
I'm not trying to incite. More on that below.
Ok, my point about the measurements was that if one is going to go to the trouble of measuring, then take the trouble to measure accurately. That was all. Remember, I wasn't disagreeing with your observation.
I was clear what was cherry-picked. The time period you selected when discussing forests and trees.
The bell curve, along with means and standard deviations, is bunk when it comes to market returns. But if standard deviations are going to be employed, then there has to be a mean to go with. This is our drift disagreement. There's a data set with a standard deviation of four units. What information does that convey? Nothing. Without an associated mean, the standard deviation has no context. Again, it's all useless, but I was getting at accuracy and consistency. This is all about being precisely wrong :-)
We'll agree to disagree about widely accepted. I figure that if I'm aware of something, and there are a whole lot of people smarter than me, then they probably know about it too. :-)
In any of my comments, I was never trying to incite. Folks need to remember that e-mails and comments, unless it's pretty explicitly laid out, have no tone other than that provided by the reader. This is an ineffective way to communicate because so much of what is transmitted by tone and body language is lost. In my view, that's why comment streams seem to degrade into back and forth attacks.
I said it might set you off because on two previous occasions you indicated my comments did annoy you. I'm really and sincerely sorry about that. The error in your probablistic thinking, my view only, is illustrated by your example in the comment stream about the biotech stock combined with the tools you're providing (I've looked at them). I touched on it when I mentioned the price movements of financials in 2008 in one of my comments. Betting on stagnation (picking up nickels near the mean) works until the day stagnation abruptly and without warning leaves (steamrolled in the fat tails); apologies for cribbing Taleb. The event path used for "actual probabilities" (correct me if that's not the price history) is only one of an infinite number of paths the price could have taken, and it is by definition incomplete. Likewise, there is an infinite number of paths the stock price can take in the future. Using tools based on variance, in a world of infinite variance, is very dangerous. Just to be upfront, I've gone down the very path you are on, or at least pretty darn close to it. We've used similar tools and calculations but, I think, looked at the output in different ways. It is compelling and tantalizing. That is why I read your original article. I truly wish you luck and good fortune along the way.
-- Don
On Jan 09 01:50 PM Crocodilian wrote:
> I did not expect to find "leptokurtotic&... on Seeking Alpha,
> and the last time I encountered a jump-diffusion algorithm was in
> an image processing seminar.
>
> But that calls to mind the problem with these statistical techniques:
> they were designed to analyze phenomena which are themselves unaffected
> by the historical predictive success of the algorithm: whether my
> face recognition algorithm works well in low light does _not_ change
> what faces look like.
>
> The historical success of an algorithm used to understand financial
> markers does, however, affect the markets its describing. It becomes
> embedded in assumptions of risk and the distributions of returns,
> for example, and the existence of many positions built on a foundation
> of assuming those historical distributions of returns, in fact prospectively
> changes the distribution of expected returns (though the modelers
> don't know it).
>
> This is part of the "dependence" of which you speak, and was ignored
> at great cost.
Anyone not believing in the article's premise, remember LTCM. Nobel Laureate Merton Scholes was working for that company, and look at where it ended up.
Here's my take on this. The entire modeling thing to me is great for estimation of probabilities of asset classes, not individual assets like stocks. And like I said, estimation only. That's it. When someone tries to be too precise, they're asking for it. You cited drift. I tend to ignore drift because it's offset by the interest rate discount that you should apply. That's not that I don't know those two factors exist. It's just that if you're going to be precise, why not go all the way? Or better, you can simply recognize that in most trading instances, it's not significant. It would be significant if you were valuing derivatives whose duration was in years. But the VIX is just a 30-day read. There’s simply no need to make things complex when something simple tells you everything you need to know. Similarly, I could have raised a stink about the Bloomberg article's use of a linear distribution instead of a logarithmic distribution. But again, over a 30-day period, it's insignificant. So I made an adjustment and omitted logarithms from the equation I posted. But that still misses the bigger point, which is no matter what adjustments you make, a model is still just an estimate!!
My point remains that there were bigger picture issues than nitpicking over drift, logarithms and interest rate discounting. I thought that introducing a new tool that let's you easily visualize the bell curve probabilities versus the real world probabilities, and making sure that my assumptions matched those used in the Bloomberg article, was more significant than getting the trivial stuff. But like I said, I can modify it to include drift. It's not that hard. Maybe I'll make a special copy for you. And I don't mean that in jest. Who knows, maybe you'll like it and endorse it.
I do want to clear up something before I go back to work though. Do you really think that I believe that betting on stagnation always works? You keep saying that it works till it doesn't. I never said otherwise. In fact, I began my prior article by saying that the market makes far more 3, 4 and 5 standard deviation moves than expected. And I'm the one who gave the example of the catastrophic biotech situation. All I ever said, with respect to stagnation is that regarding one standard deviation moves specifically, in the S&P 500 specifically, over the past 18 years specifically, the bell curve assumptions misjudged the odds. I then provided readers a mechanism to test different standard deviations to see how different tails stacked up. That’s it. I am not a partner in Capital Decimation Partners, LLP.
Lastly, whether you meant to or not, you did raise my ire when you said I'm "falling into a trap" and "you are making some serious errors". Well, I'm not. Prior to that, we differed on drift. But then, out of the blue, you say I’m falling into some traps. I asked you to explain, and you said that actual probability distributions are just one of an infinite number of paths that a stock could take. That is true … But here’s what’s really important, at least to me -- I never said otherwise!
All I ever said is that the bell curve and the models based on it are wrong, and here’s a free tool that uses past historical price movements to prove it! For you to take the general premise of my articles, which by definition don’t include everything in my mind, and then say I’m perpetuating a myth, and then turn it into a lecture on me falling into a trap is condescending. And to go even further and say I am making some serious errors, when the error you cite is me combining the biotech example with the probability analysis tools on my web site, and somehow concluding that based on a bunch of numbers, I believe that the improbable becomes impossible—which I never did—is, well, that’s just wrong. Because I never said it, wrote it, or thought it!
I could go on and on, and qualify my remarks even further. But I’ve got a cold one on my desk and not in my hand, and it’s begging to be consumed. Plus, I’m sure this thread is boring to everyone else. And, I don’t want to give away all my secrets!!!
But more important. I don’t think you and I disagree on anything consequential. I just ask, be aware of your words. Saying I’m perpetuating a myth, falling into a trap and making serious errors are inflammatory, just isn’t necessary.
Stay in touch.
-- Don
I'm one ahead of you. I have a cold one in hand. And, I agree, the thread is probably getting boring for others. I'd ask the same of you regarding inflammatory comments. You're accusing me of some things I haven't done. We could go back and forth, but what's the point. C'mon, Don, give me a smile. It's Friday. ;-)
We started with a difference on drift. And we do agree on many things. Like you, I don't intend to give away all my secrets. I'm a big fan of Taleb's writings, but there's a subtle hole in his work. I suspect he knows it, but I'm not sure he'd sell as many books if he acknowledged it. The funny thing is that he rails against means and standard deviations, and yet thinking in means and standard deviations is exactly what causes it. My spreadsheet tools which did more or less exactly what I think you're doing contained this same hole. Perhaps you've already realized it. Here's a hint, and keep the normal probability vs actual probability methodology in mind: take a mean and standard deviation for returns of any time period. Now consider what just happened to volatility. Now consider the observation, which Taleb, Mandelbrot, you and I (we're in good company!) still make that there are far too many 4th, 5th, and 6th standard deviation moves. It hit me one day while I was mowing the lawn. I threw out a lot of spreadsheets. Now, you may draw a wrong conclusion about my thinking from the way that's written. I'm not saying that there aren't many more large moves than there ought to be under a normal distribution. I'll leave it at that. Cheers!
My personal belief is that you can never, ever predict the market so why waste your time trying to do that? You CAN, however, always predict how people will react to a situation. Humans just don't change. I recall in one of your comments you mentioned extreme, unpredictable events (something with the ASCO convention). An example like that could be explained by investor overconfidence in the company for finding a breakthrough cure in cancer (otherwise why would they hold the conference to begin with?) Please keep in I'm taking a pretty general view here, just to get my point across.
Let me know. As someone who also doesn't believe in statistics, I'm really curious about what you think about behavioral finance trading...
So I'm going to offer a little bit more advice than the question asked. Because it's what kept me profitable in 2008 even though I did nothing but take bullish and non-directional option trades all year long. Here's what's key, at least from a trading perspective (not an investment perspective). You can be wrong on a huge number of trades and right on a few, and still make money, as long as when you're wrong, your allocation to those losers is small and your allocation to the winners is big.
Now what's the solution to that? I don't have the answer for this forum. But I'll just say this. We had larger allocations to trades through the entire year till September, then scaled back a lot, then picked back up on the allocation in December. This month, we've scaled back again. Had we gone with a persistent allocation throughout the year, we would have gotten hammered in September, October and November.
In my mind, you don't need any directional "indicators" to make money in the market, at least when you're trading. [Investing is another matter.] Indicators certainly help. And I use them. But what I've found that is far more critical is that you have proper allocation schemes. That's where the real difference comes from.
I know I went on a tangent there. So I hope I answered your question.
-- Don
On Jan 10 12:35 AM Gisi wrote:
> Don- just out of curiosity, what is your view on trading based on
> behavioral indicators?
>
> My personal belief is that you can never, ever predict the market
> so why waste your time trying to do that? You CAN, however, always
> predict how people will react to a situation. Humans just don't change.
> I recall in one of your comments you mentioned extreme, unpredictable
> events (something with the ASCO convention). An example like that
> could be explained by investor overconfidence in the company for
> finding a breakthrough cure in cancer (otherwise why would they hold
> the conference to begin with?) Please keep in I'm taking a pretty
> general view here, just to get my point across.
>
> Let me know. As someone who also doesn't believe in statistics, I'm
> really curious about what you think about behavioral finance trading...
On this topic there was an interesting and self serving document published this week by
two luminaries from the world of financial engineering - Paul Wilmott and Emanuel Derman. It was called The Financial Modelers' Manifesto and was mailed to anyone on Paul Wimott's mailing list and is available at his website and elsewhere.
The short document is worth reading not so much for its discussion of the limitations of applying mathematical modelling and statistical techniques to financial time series analysis but, more importantly, because it highlights the manner in which a lot of quants that are now proclaiming their innocence were previously guilty of the hubris and irresponsibility that lead directly to the massive mis-pricing of risk in financial derivatives.
Here is an excerpt from the Manifesto
MODELERS OF ALL MARKETS, UNITE!
You have nothing to lose but your illusions.
The Modelers' Hippocratic Oath
~ I will remember that I didn't make the world, and it doesn't satisfy my equations.
~ Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.
~ I will never sacrifice reality for elegance without explaining why I have done so.
~ Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.
~ I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.</i&g...
It is the last part of the Oath that reminds one of that old expression about stable doors and horses bolting.
One of the most refreshing things that could emerge from such high powered quants discovering God would be that it might (although I doubt it) lead to a radical rethinking of the way that finance is taught
Future MBA’s should be taught something more historically and philosophically oriented on the nature of risk, rather than the same mathematics that is useful in the physical sciences but is more or less useless when applied to the world of financial economics.
Being a relatively new private boater, I was fascinated by references to “rogue waves” and researched the topic. Basically, for hundreds of years naval architects and marine engineers did not believe sailor’s tales of giant “rogue waves” and relied on statistical models of wave heights based on wave physics and weather conditions to design ships. Yet, every year many more ships were lost than could be explained by the wave models. Sometimes, ships just vanished in apparently fair weather.
Finally, some data came to light that could not be easily ignored and the problem was actually studied scientifically and systematically in the late 70s or early 80s I think it was, including by satellite observation, and it was confirmed that giant waves that can sink huge ships can and do form in the open ocean at a frequency that greatly exceeds what the statistical models predicted. So, the statistical models and physics used did not accurately model the physics of wave formation. The data proved it.
Apparently, the same may be said about statistical models and markets. Within certain limits, all else being equal, the statistical models work pretty good. The problem is the “all else” doesn’t always stay equal. A few obvious reasons are: market manipulation, rules changes, war, and availability of credit.
My analogy is that statistics does a great job of predicting the outcome of a spin of the roulette wheel, until the house decides to clean up and hits the “00” switch three times in a row. The problem is figuring out who the house is at any given time and when they might trip the switch. And, sometimes it is just a massive change in investor psychology that takes place during a major trend reversal with no one in particular pulling strings.
Because there is no defense against every possible Black Swan event, money management is a key element of investing: not putting too many eggs in one basket, cutting losses, taking partial profits, not going “all in”, hedging bets, etc.
Hope you and CM settle down. You're both smart and interesting but the thread was getting boring.