Tracking Error In Exchange Traded Funds

Exchange-traded funds have emerged as the asset allocation tool of choice among sophisticated investors. The reason is simple: ETFs allow investors to "buy the benchmark" when they want, how they want, and in any size they want … from the comfort of a regular brokerage account.

In an ideal world, ETF investors would get exactly what they thought they were buying. That is, if an ETF's benchmark was up 100 percent for the year, a $100 investment would become $200. Alas, that is practically never the case. Index portfolios, no matter how well run, always suffer from some amount of "tracking error." Tracking error is the difference between the performance of a fund and the performance of its underlying index. Generally speaking, the less of it, the better.

Morgan Stanley Global Wealth Management recently surveyed the tracking error of 204 U.S.-listed ETFs for the calendar year 2006, including substantially all equity and fixed-income funds with twelve months of data as of December 31, 2006. The funds were divided into eight different asset categories:

Major Market: Major market U.S. equity ETFs tracking indexes like the S&P 500, DJIA, etc.

Dividend: ETFs tracking high-yield stock indexes, i.e., the Dow Jones Select Dividend Index.

Custom: Uniquely defined ETFs tracking indexes like the Intellidex Dynamic OTC Index and the Lux Nanotech Index

Style: Major U.S. equity market ETFs tracking the nine "style box" categories.

Sector: All U.S. sector- and industry-level funds.

International: International funds tracking indexes like the MSCI EAFE and S&P Latin America 40.

Global: Global ETFs that hold both U.S. and International assets, including sector funds.

Fixed Income: U.S.-focused bond ETFs tracking indexes like the Lehman TIPS Index.

The study excluded all commodity focused ETFs, as well as HOLDRs and other static-basket products.

Measuring Error
Morgan Stanley took the broadest possible definition of tracking error for this study, measuring it in absolute returns versus the benchmark index. With these grey-colored glasses on, it is just as important to see that a given fund over performed its benchmark as under performed. The idea is to measure tracking risk, and the concept of alpha is as largely irrelevant when measuring risk. The study looked at the simple absolute variance between the performance of the relevant benchmark and the end-of-day NAV of each ETF.

Figure 1 showcases the extreme and average tracking errors Morgan Stanley found for its eight investment categories.

FIGURE 1

This data offers two key insights.

First, the average tracking error is quite good: no investment category missed its mark by more than one percent for the 12 months ending December 31, 2006. Furthest from the mark were the group of eight Custom ETFs, representing both a small sample and the most eccentric benchmarks. Not surprisingly, Fixed Income ETFs came closest to their benchmarks, missing by just 9 basis points on average; this reflects both the low fees associated with those funds and the high liquidity of their markets.

The second interesting conclusion from this most basic look is the phenomenal variability of tracking error. This result was diluted from an investor's perspective last year, as several of the biggest outliers actually represent positive tracking error. The Custom group, for example, stands out because two funds-the PowerShares Lux Nanotech (AMEX: PXN) and Wilderhill Clean Energy (AMEX: PBW) ETFs-outperformed their benchmarks by 149 and 229 basis points, respectively. These upside surprises were offset by similarly dramatic under performance in other areas: the Sector group's nadir is pinned by the iShares DJ US Telecom ETF (NYSE: IYZ), which underperformed its benchmark by a stunning 446 basis points.

Prime Suspect: Fees
The most obvious explanation for negative tracking error is baked into the funds from the start: expenses. A fund's management fees are taken directly out of net returns, so the higher the fee, the larger the tracking error (in most situations). Fortunately, this component of tracking error is both predictable and transparent.

Figure 2 shows the relationship between fees and average tracking error for the eight investment categories.

FIGURE 2

Even a casual glance reveals some obvious conclusions. The first is that the more established and larger-cap the benchmark, the more clearly tracking error can be explained by expense ratio. Major Market, Dividend and Style funds (all based on highly liquid U.S. equities) all showed a difference of less than 10 basis points in non-fee tracking error.

The primacy of fees is further born out when we look at how fund families do vs. each other, regardless of asset class. Vanguard's ETFs, with the lowest average expense ratios in the Morgan Stanley survey, showed the lowest tracking error overall: just 28 basis points off their benchmarks. At the other end of the spectrum, the PowerShares ETFs have an average expense ratio of 59 basis points, and average 71 basis points in tracking error.

The standout, however, is fixed income. Fixed Income ETFs have the lowest tracking error in the sample universe. If you bought a corporate bond ETF last year, you got very close to the underlying index's return. In fact, fixed income is the only category where tracking error was less than the management fee in 2006: for instance, the iShares Lehman TIPS ETF (NYSE: TIP) earned back half of its 20 bps management fee, only trailing its index by 10 basis points.

The Devilish Details: Diversification And Optimization
Despite their apparent simplicity, there is real skill involved in running an ETF, including managing index changes and deciding how closely to mimic the index portfolio. Some funds use a replication strategy, buying exactly the same stocks at the exact same weights as the underlying index; this may raise trading costs, but it generally limits tracking errors after fees. Other funds use "optimization techniques," buying a subset of the index's stocks in the belief that they will provide similar performance to the full portfolio, at a lower cost to trade. The degree of optimization influences the tracking error risk.

One reason that Broad US equity ETFs track very well, for instance, is that they can mirror the (generally liquid) components of their indexes very closely. For example, using Morgan Stanley's example, the Vanguard Total Stock Index ETF (AMEX: VTI) holds 3,795 of the 3,874 stocks in the MSCI US Broad Market Index. This, combined with a sector-low management fee of 7 basis points, helped VTI to have the lowest tracking error of any major U.S. stock ETF last year, at just 5 basis points.

At the other end of the spectrum, narrow sector funds have a problem, because Securities and Exchange Commission [SEC] diversification requirements can tie the hands of ETF managers. The basic ground rules for all mutual fund (including ETFs) are set in stone:

• No single security can be more than 25 percent of the portfolio
• Securities with more than a 5 percent share can't make up more than 50 percent of the fund.

For narrowly focused ETFs, this can make it impossible for the funds to match their benchmark indexes. The fund with the highest variance in Morgan Stanley's survey makes this point clearly.

The iShares Jones US Telecom ETF (NYSE: IYZ) trailed its benchmark last year by 4.46 percent-a huge margin for an ETF. The reason is simple: AT&T makes up 48.2 percent of the index, while it cannot be more than 24.99 percent of the fund. As a result, AT&T's weight in the portfolio is reduced to 25 percent or less, and the remaining 23 percent of the portfolio is assigned to other stocks. This causes massive under- and overweighting.

IYZ trailed its benchmark last year because AT&T had a very, very good year: the stock rose 53 percent, far more than other stocks in the portfolio. The performance meant that fund managers were essentially helpless to do anything but watch as their portfolio trailed its benchmark.

Figure 3 shows the weights of companies in the Dow Jones U.S. Telecom Index and the related ETF, as of January 31, 2007.

FIGURE 3

These issues are further exacerbated in some international portfolios which, being both concentrated and cap-weighted, result in heavy concentration. The iShares MSCI Mexico fund (NYSE: EWW), for instance, tracks a benchmark in which the top five holdings account for over 70 percent of the index-forcing the manager to diversify into smaller issues. In the case of EWW, this ended up benefiting shareholders, as the re-weighted ETF actually outperformed its index by approximately 2.5 percent, thanks to strong returns for Mexican small cap stocks.

The Intangibles
Cash Management
An ETF, like any mutual fund, is challenged to minimize cash as much as possible, and despite the intermediating roll of market makers, cash does slip into a portfolio, whether through dividends, overnight balances, trading activity or securities lending. Since no major index is constructed with a cash component, the presence of any cash in the portfolio will show up as tracking error. How big an impact cash has is largely determined by fund composition. The iShares DJ Utilities ETF has a large cash position (10-20 basis points) from the high dividend payouts, whereas major market funds often have 4-5 basis point cash positions.

This excess cash can become a form of "dividend drag," where the lag between receiving and reinvesting cash causes variance. From experience, we can assume that this is usually a small contributor, but an unknown one, and thus a source of risk.

Securities Lending
One source of return not present in any index is the sometimes controversial practice of securities lending. Major index players such as Barclays Global Investors [BGI] and State Street Global Advisors have long been a source of securities for brokerage companies looking for inventory to service their short selling customers (particularly hedge funds). Since large index portfolios are often the only place where large concentrations of more thinly traded stocks exist, they're able to run quite lucrative stock loan desks.

In a stock loan, the management company takes collateral from the brokerage firm and earns returns on that collateral. In the case of Barclays [BGI], the revenue from that collateral investment is split evenly between the fund and Barclays; in other firms, the revenue may accrue fully to the fund. While there is theoretically counterparty risk in stock loans, in practice, it is essentially 100 percent upside for both investors and management companies.

Trading Efficiency
Lastly, despite the static nature of target benchmarks, the day-to-day life on the trading floor of an ETF shop is anything but sedate. A quick glance through the annual portfolio disclosure of any ETF will give you a snapshot into that ordered chaos. Positions are constantly being modified, optimizations enacted, dividends reinvested, and cash collateralized. As with any human endeavor, some people are better than others at this execution, and this skill and efficiency results in reduced costs and increased returns for even the most passive portfolio.

Averages Don't
While Morgan Stanley's study provides insight into the statistical underbelly of the ETF market, it should be seen as only one of many factors influencing actual real-world investor experience. The largest missing component is the individual execution of any position. Commissions and bid/ask spreads can be either negligible (for a large institutional investor) or quite substantial (for a small investor in a full-service brokerage account). Further, few investors actually buy their ETF shares at the end of the day, as the NAV for the Index (and the ETF) are recorded for posterity. While potentially a minimal contributor to performance, shares purchased on a highly volatile day-say, during a news-driven 5 percent dip in an index on an otherwise flat trading day-can have a significant impact on long-term performance.

In short, no investor's experience is "average." Rather, each investor will have a unique pattern of returns. The lesson here is to keep a focus not just on the index to which you've allocated your assets, but to the inherent risks, however small, associated with the vehicle you've chosen to use.

ETFs have been a tremendous financial innovation and a powerful tool for all kinds of investors. But line anything else, they can't be perfect.