"I think the idea that everyone can have wonderful results from stocks is inherently crazy. Nobody expects everyone to succeed at poker."
Being "better than average" is not enough to beat the market. In other words, you can be in the top half of stock pickers based on a Gaussian distribution and still underperform the returns of the broad market index.
Everyone thinks that they are above average
Everyone that I have ever met truly believes that they themselves are above average, but we can't all be right. Luck and the invisible hand may give a minority of stock pickers a competitive edge, but it cannot improve the results of investors as a group.
The market is the sum of its participants
The idea that investors as a group must be average can be traced back more than a century to a man named Louis Bachelier, who in his PhD thesis at the Sorbonne in 1900 presented the notion that, "past, present, and even discounted future events are reflected in market price". Thereby laying the foundation of what has come to be known as the efficient market hypothesis.
As Nobel Laureate William Sharpe explained so well, by definition investors as a group will earn the market's return before costs, and will lag the market's returns after costs. This is true for both the stock market over all, and the individual sectors that it is comprised of.
Conflating the mean with the median
When investors talk about performance they are thinking about a normal, or Gaussian, distribution, wherein half of investors are above the average and half are below. By definition they are absolutely correct, 50% of investors are better than the other 50%. The performance of investors follows a normal distribution.
However, stock performance does not follow a normal distribution, it follows a log-normal distribution pattern. Stock price distributions exhibit what is known as a heavy-tail, sometimes called a fat-tail. Stocks must have a "fat" or "heavy" tail because prices can only go down to zero, but may go up infinitely. Thus, extreme outliers on the positive side of the distribution curve will skew the average return above the median, as seen below.
Therefore, it is entirely possible that you may be an above average investor as defined by a normal distribution of investor performance, but still underperform the index.
The proof is in the pudding
Seeking Alpha contributor Rick Willis tested this distribution data using the real results of the Dow Jones Industrial Average and found the following
"For 2012, the median return for the 30 stocks was 11.22%. The average return was 14.13%. For the four-year period from 2009 through 2012, the median return was 61.44% and the average return was 69.84%. For the 10-year period from 2003 through 2012, the median return was 106.57% and the average return was 134.66%. This is far from a conclusive examination, and any rigorous test of this hypothesis should include a much larger sample of stocks and many more periods. But this quick test does show that the average value exceeds the median value, and this disparity grows with time."
What is true of the part is true of the whole
Just as the prices of the index components are distributed on a log-normal basis, so are the returns of the indices themselves. According to data compiled by forecastchart.com, during the 10 years period ending February of 2013, the Dow Industrial Average had a return of 78%. The best performing index during the period was the Mexico IPC Index, with a return of 646%. The worst performing index during the 10 year time period was the Nikkei 225 Index with a return of 34%. The median return for the indexes during the last 10 years was 101%, while the average return for all stock market indexes over that time frame was 137%.
Putting thought into action
Because market returns tend to fall along a log-normal distribution, index investors who earn the market average return are going to outperform active investors, who as a whole are going to earn returns closer to the median. What is true of stock picking within the index holds true for the indexes themselves. Therefore, we should take this information to its logical conclusion and own the entire world market.
Vanguard Total World Stock Index (VT) is a one stop fund to own the entire global marketplace. No matter what region or industries outperform in the coming years and decades, this funds broad portfolio ensures that it will own them and investors will get their fair share. If you buy into the idea of letting markets determine your holdings, letting markets decide your country allocation is a logical extension of that. Given that where a company is domiciled doesn't really say much about where a company actually does business, the global index approach makes perfect sense for index investors.Summation
Most of the time when investors speak about beating the market, they are implicitly conflating the mean with the median. In their minds, being in the top half of investors (being above average) is equal to beating the market. However, because stock prices follow a log-normal distribution, being an above average investor is not a guarantee of earning returns above the rate of the market. By capturing the expected returns embedded in the market as cost efficiently as they can, index investors always earn "above average" returns.