Back during World War II, statistician Abraham Wald was tasked with helping the Allies decide where to add armor to their bombers. The hope was that this extra protection would help minimize bomber losses to enemy anti-aircraft fire. The top brass of the Allied army thought the answer was obvious: just look at the bombers that returned from missions, and add armor to the areas that showed the most damage. But Wald disagreed. He explained that the damage actually revealed the locations that needed the least additional armor; in other words, it's where a bomber could be hit and still survive the flight home. Wald's solution was counterintuitive. He recommended adding more armor to places like the engine where there was no damage, because that's where the bombers that didn't make it back were hit. This simple advice would end up saving the lives of thousands of Allied air crews.
Typical Damage Patterns on Returning Bombers
Source: A North Investments
The bomber problem is a classic case of "survivorship bias" - the tendency to only consider information that's presented to us (e.g., bombers that survived), and ignore absent information that may be extremely relevant (e.g., bombers that got shot down). Focusing on the former and ignoring the latter distorts the way you think and make decisions. It's a bias that frequently arises in all kinds of contexts. And once you're familiar with it, you'll be primed to notice it wherever it's hiding.
Like health and longevity advice. We look to old people on guidance for living a long life when we should really examine those who died early to learn what to avoid. I recall watching a documentary about centenarians (100+ year olds) who claimed that the key their longevity was smoking and drinking every day. The non-statistically minded (which includes most of us) will misinterpret this as proof that smoking and drinking isn't that harmful, not realizing that centenarians represent the lucky few who won the genetic lottery. There's a much larger pool of people who made the same poor health choices and didn't live long enough to appear on television; so the unusually lucky few tend to stand out and, hence, receive the most attention.
Survivorship bias also skews our understanding of the past. Take the widespread and highly romanticized belief that old things (pick one: cars, TVs, toasters, etc.) were made better than they are today. The much more likely truth is that 99% of old things were poorly made and are now rusting out of sight. The few things that did manage to survive intact were the ones that were well made. That doesn't mean everything was. The same goes when it comes to music. Songs that leap from memory when someone mentions a decade like the 1980s tend to be songs that became hits. As a result, good songs begin to represent 1980s music because we still listen to them, even though the vast majority of music produced during that decade was less than memorable. Our childhood memories work much the same way. We tend to remember the good times and forget the bad, deceiving ourselves into thinking that the "good old days" were far better than they really were. No wonder the present never seems as good as the past.
Even religious beliefs are affected by survivorship bias. Consider the story told by Marcus Tullius Cicero about the atheist Diagoras. He was shown the painted portraits of faithful worshipers who prayed and were later saved from a shipwreck. The implication was that praying protects you from drowning. Diagoras asked, "Where are the portraits of those who prayed, then drowned?" There were none. Dead worshipers, like the downed bombers, can't advertise their experiences, so they get excluded from the sample. This is how people get fooled into believing in miracles (which are nothing more than positive low-probability events). Let's say a disease is 99.99% fatal and 1,000 people get it. The single survivor will surely see his recovery as a miracle; of course, the reality is that the statistics of the situation simply dictated that "someone" would survive. The lucky survivor gets to stick around and tell his miraculous story; the 999 non-survivors, being dead, can't tell their non-miraculous ones.
Something similar occurs in the investment industry. It claims that some people are extremely skilled, since year after year they've outperformed the market. They'll identify these "investment gurus" and convince you of their abilities. But a simple thought experiment can show that it would be impossible to not have these gurus produced just by luck. Imagine you had thousands of money-managing chimpanzees picking stocks at random. If every year you fired the losers, leaving only the winners, eventually you'd end up with long-term steady winners. Since all you see are the handful of survivors, you'll be led to believe that random stock selection is a good investment strategy, and that some chimpanzees are considerably better investors than others. Plus, since chimpanzees charge lower fees than their human counterparts (bananas are inexpensive), you might even be tempted to let one of these hairy creatures manage your portfolio. I wouldn't recommend it.
Even though human money managers employ more sophisticated investment strategies, it's still easy to get fooled by survivorship bias. Given the multitude of different strategies, some are bound by pure luck, even over long periods of time, to produce superior performance even if they don't genuinely possess predictive power. And it's this small subset of surviving strategies that attract the most attention and investment capital. Consider the famous Super Bowl Indicator, which says stocks go up in years when a team from the NFC wins and down when an AFC team wins. It was right 63% of the time between the first Super Bowl in 1967 and 2014. In years where an NFC team has won, however, the indicator's accuracy improves to 88%. Sounds impressive. Unfortunately, as is the case with many popular investment strategies, the indicator has no predictive power - it's simply a case of spurious correlation (just like ice cream sales and forest fires are correlated, but neither causes the other). Making investment decisions based on such random relationships is how you go broke.
S&P 500 Returns as Predicted by the Super Bowl Indicator
Note: The 2015 Super Bowl was won by the New England Patriots (an AFC team), which means that 2015 should be a down year for stocks . . . so far it appears to be the right call.
Source: A North Investments
Some of today's most widely used investment strategies are nothing more than Super Bowl Indicators in disguise. Take Warren Buffett's focused value investing, which involves betting heavily on a few high quality, undervalued companies. He's made tens of billions of dollars following this approach, and it seems to work for others too. Forbes' wealthiest people list is almost entirely made up of individuals who, like Buffett, were rewarded for putting all of their eggs in one basket. You simply don't become as rich as Facebook's (FB) Mark Zuckerberg or Microsoft's (MSFT) Bill Gates by holding a well-diversified portfolio. But the reverse is also true - holding an under-diversified portfolio probably won't make you that rich either (if anything, it'll help you go bankrupt). Just look at the cemetery. The graveyard of failures is full of unlucky people who, just like the population of life's lottery winners, put all of their eggs in one basket. Ignoring them is like ignoring the bombers that were shot down, it's financial suicide.
Which brings us to the whole notion of success itself. Numerous studies on the topic follow a similar methodology. They take a population of suit-and-tie-wearing hotshots and look at what they all have in common: courage, passion, risk taking, vision, and so on, and infer that these traits are the "secret to their success." To recognize the flaw here, simply look at the cemetery of failed persons. What traits do they all share in common? Here's a hint: courage, passion, risk taking, vision, etc., just like the population of hotshots. This is difficult to see because failures don't appear on television and on the covers of magazines. They don't write books and memoirs. They don't travel the world giving seminars and lectures. In short, nobody's interested in what they have to say (even though they can teach you some useful tricks like "what not to do" and "mistake to avoid"). There may be some differences in skills, but what truly separates life's winners and losers is plain old luck.
The moral here is to be careful from whom you seek advice, because advice-giving is a monopoly run by lucky survivors. To paraphrase the famous psychologist Daniel Kahneman, stupid decisions that work out well become brilliant decisions in hindsight. The things people like Steve Jobs or Mark Zuckerberg or Bill Gates did right are like the damaged bombers that managed to survive the flight home. The much larger pool of people, those who made equally risky bets and failed, exit the sample. This gives us a distorted view of the odds of success. So, before you emulate a famous entrepreneur by dropping out of college and starting a business in your parents' garage, ask yourself this question: How many people have done this same exact thing and failed? Way too many to count. No one remembers or cares about these losers and their unsuccessful companies. For every wealthy startup founder, there are thousands of other entrepreneurs who end up with only a cluttered garage. Surviving those statistical odds is nearly impossible.
In Plato's "Allegory of the Cave," he describes a cave where prisoners are chained, unable to see anything, except shadows on the wall before them. The prisoners believe the shadows to be reality. However, one prisoner is freed and brought outside for the first time. At first he's blinded by the brightness of the sun, but after his eyes accustom he realizes that the shadows were only fragments of reality. The point here is that what we see isn't always the whole truth. Spending your life only learning from survivors, reading books about successful people, and poring over the history of companies that changed the world is like being imprisoned in Plato's Cave - all you get is a biased and incomplete view of the facts. However, leaving the cave and acknowledging the other side of the equation (i.e., the non-survivors, losers, failures, etc.) prevents you from getting fooled by survivorship bias.
