By Michael Rawson, CFA
With so much risk in the markets, when we take a passive position in an index to form the core of our portfolio, we want to be sure that the fund company is actually doing what it promises it will do. One way to evaluate how closely a fund sticks to its benchmark is to look at tracking error. In this article, we compare two S&P 500 Index-tracking ETFs with a large sample of index mutual funds to see which products do a better job of tracking the index. Does paying a higher expense ratio result in lower tracking error or better performance? We also discuss different ways to define tracking error. We conclude that the S&P 500 tracking ETFs have performed as promised, with a lower tracking error and better performance than almost all index mutual funds. However, the granddaddy of all index mutual funds can still teach the new kids on the block a thing or two about efficiency.
We analyzed five years of weekly returns for 61 index mutual funds and two ETFs that track the S&P 500. The following table shows just the five mutual funds with the lowest tracking error and both ETFs. Two Vanguard mutual funds had the lowest tracking error, but one is an institutional product with a minimum investment of $5,000,000 while the other has a higher expense ratio, which leads to lower performance.
click to enlarge
The next lowest tracking error product, iShares S&P 500 Index ETF (IVV), has no investment minimum (beyond the price of one share) and only a 0.09% expense ratio. This was the same expense ratio as the other ETF in our survey, SPDR S&P 500 (SPY). That product is structured as a unit investment trust, rather than as a 1940 Act fund, which prevents it from reinvesting dividends back into the fund. The inability to reinvest dividends leads to a cash drag that creates tracking error, particularly in volatile markets. Most mutual funds also have a cash drag due to keeping cash on hand to fund redemptions. For SPY, this leads to a slightly higher tracking error than IVV, but it is still less than half the average fund tracking error. Both ETFs had expense ratios well below the average index fund (which was 0.38%) as well as tracking error much lower than the 0.29% average tracking error. Perhaps most importantly, both ETFs had better performance than the 0.28% average fund yearly loss.
Is Cheaper Better?
With this data, we can address the question of whether the expense ratio matters. For instance, out of the 63 funds (grouping the ETFs with the mutual funds and including institutional funds with high minimum investment requirements), the 10 funds with the lowest expense ratios charge an average expense ratio of just 0.11% and underperformed the benchmark by an average of just 2 basis points per year with a tracking error of 0.22%.
On the other hand, the 10 most expensive funds had an average expense ratio of 0.85%, underperformed the bench by an average of 0.68% and had nearly double the tracking error at 0.41%. Both the return and the tracking error were impacted by the expense ratio and this relationship was statistically significant.
Thus, when it comes to S&P 500 Index funds, cheaper is better. Many analysts claim that the high expense ratio causes a high tracking error and that the expense ratio should somehow be added back to reduce tracking error. This is not true in our analysis, where the expense ratio would reduce a fund's return but have no direct effect on tracking error.
ETFs are often said to be tax efficient due to the fact that the in-kind creation and redemption process allows an ETF to exchange low-cost basis shares for shares of the ETF, washing away the tax basis. However, this is not the only way to save on taxes.
Vanguard has a lot of experience managing for tax efficiency and is able to achieve economies of scale given the size of their fund. (Vanguard 500 Index Investor (VFINX) has $86 billion in assets). Fans of founder Jack Bogle, Bogleheads as they are known, tend to be buy-and-hold investors, so the fund does not incur the same kind of hot money redemptions that we might see in other funds. Over the past five years, the fund has had a tax-cost ratio of just 0.31%, which compares favorably with the 0.69% for IVV and 0.67% for SPY. This means that investments in these funds held in taxable accounts would have lost that amount of assets each year, thus reducing return. Certainly the flat returns over the past 10 years have helped prevent capital gains, but going forward, VFINX has a larger potential capital gains exposure at 9% versus negative exposure for both of the ETFs. For more on the tax efficiency of Vanguard's products, take a look at this video featuring Vanguard CIO Gus Sauter.
Tracking Error Defined
Tracking error can be defined in a number of ways. Perhaps the simplest is to look at the amount by which a fund underperformed its benchmark. While this is what investors care about most, it says nothing about the probability distribution of how that fund might perform in the future. After all, we are concerned about risk, so a good definition of tracking error should tell us something about the risk our fund manager is taking.
You have heard the saying, past performance is no guarantee of future results. While that may be true for a manager's ability to beat an index, it turns out that overall risk is relatively stable and somewhat predictive of future levels of risk. The excess return of a fund is the fund return minus the benchmark return over a unit period of time, such as daily or monthly. The time series of excess returns will include both the positive and negative values from when the fund beats or trails the benchmark, whereas underperformance just looks at negative values. If we define tracking error as the standard deviation of those excess returns, we can now judge the variability of the fund. For example, if the standard deviation of a fund is 1%, then we can assume that the fund should be within 1% of the benchmark 68% of the time, and should lag the benchmark by more than 2% only less than 3% of the time.
One criticism of this approach is that it penalizes for either beating or trailing the index, while beating the index seems like the kind of risk we want to have. While that point is valid, it should be noted that in an efficient market, managers would not be able to beat the index without taking risk. While it may pay off in one period, it may not in the next and we should at least be aware if the manager is deviating from his stated objective. While not perfectly symmetrical, stock returns (unlike alternatives like options) can be approximated by a normal distribution so the positive side of the standard deviation looks similar to the negative side.
A slightly more complicated definition of tracking error is the standard error from a regression of fund returns on the benchmark. This will break fund performance into three categories: manager skill (alpha), market risk (beta), and tracking error (standard error). This approach has the added benefit of adjusting for the expense ratio of the fund as a reduction in the alpha and any leverage in the fund through the beta.
Disclosure: Morningstar licenses its indexes to certain ETF and ETN providers, including Barclays Global Investors (BGI), First Trust, and ELEMENTS, for use in exchange-traded funds and notes. These ETFs and ETNs are not sponsored, issued, or sold by Morningstar. Morningstar does not make any representation regarding the advisability of investing in ETFs or ETNs that are based on Morningstar indexes.