You don't have to look very far before coming across the advice that individual investors should simply invest in index funds rather than hire financial advisors to manage their money. Generally speaking, financial advisors don't "beat the market" and they charge fees for their services, and so the theory is that most investors would be better-off buying an index fund that holds the market rather than attempting to pick stocks on their own.
This seems logical: Owning a major stock index like the S&P 500 large-cap index provides an investor with a diversified portfolio of large-cap stocks with steady dividends and minuscule fees. Had you invested in the S&P 500 (SPY) in the summer of 2003 and reinvested dividends, the return on your investment over the last 8+ years would have been 5.8% per year.
While this isn't the 9-10% returns that are touted as the long-term average for stocks, market conditions over this period have been difficult, with 2 large recessions factored-in. After all, SPY is one of the biggest index funds on the market, with $99 billion in assets and trading a staggering $23 billion per day. It's easy to assume that it must be a good investment given it's popularity.
However, if we look at the S&P 400 mid-cap index (MDY), we see that it has returned 9.6% per year over the same period. Conventional wisdom is that mid-caps have more "headroom" to grow than large caps, but this doesn't really tell the whole story.
Both the S&P 500 and S&P 400 are market-cap weighted indexes, meaning that stocks are weighted by their market capitalization. However, within large-cap stocks, the largest has approximately 200x the market cap of the smallest, while within mid-cap stocks, this difference is 5x. Of course, this varies from day-to-day, but the market cap weighting of large-cap stocks is more extremely skewed than market cap weighting mid-cap stocks.
The problem with market-cap weighting large-caps is that the largest of the large caps really do under-perform the market. The chart below shows the (back-tested) performance of various market-cap segments since the summer of 2003.
The chart clearly shows that the top 5% (and possibly the top 15%) of market capitalization under-performs the rest of the market, but the real tragedy is that this is the same segment of the market that is weighted most heavily in the S&P 500. These are the so-called "blue chip" stocks like General Electric (GE), Exxon Mobil (XOM), Microsoft (MSFT), IBM (IBM), Google (GOOG), etc.
Now, one theory behind investing in blue-chip stocks is that these are big companies that are likely to weather recessions better than the rest of the market. The chart below shows how the various market-cap segments performed during the Great Recession, from the peak to the trough. Any down-side benefit (if indeed there is one) is certainly lost in the noise.
While there are no funds that avoid the top 5%-15% of the market, the effects of these underperforming stocks can be minimized by equally weighing the S&P 500 instead of weighting by market cap. Fortunately, there is an S&P 500 Equal-Weight index, and a fund that tracks it: RSP. As the chart below illustrates, this avoids most of the performance problems with SPY by not skewing the index toward underperforming segments of the market.
While mid-caps still outperform large-caps, it is clear that large-caps can also perform well, so long as the index is not biased toward underperforming stocks.