Contrarian Investing IV - Market Timing And Sector Rotation To Profit From Investor Sentiment

by: Robert Keyfitz

"A Contrarian Guide to Market Timing" found that over the past decade a strategy of buying SPY for a limited period after observing the American Association of Individual Investors (AAII) bullish sentiment index below 25% would have outperformed the alternative of buying and holding SPY throughout. Interestingly, though, the strategy made relatively little use of the bullish sentiment index: only around 5% of observations triggered a (very simple) action. This article explores whether a more elaborate system of sector rotation using ETFs and driven by signals from a wider range of bullish sentiment could do better yet. A provisional answer is yes, as illustrated by the blue line in Figure 1. However, this is still work in progress, offered as much as anything in the hope of eliciting feedback from other interested readers.

Since the new strategy is not strictly contrarian in the sense of betting against the crowd, for lack of a better name I'll refer to it as "active, sentiment-based." The variant simulated in Figure 1 delivers a whopping 18.4% average annual return over the period 2002-12, three times the 6.4% of buy-and-hold SPY. A disadvantage is that it is transactions intensive, averaging around 25% turnover per week. However, Figure 1 assesses a transactions charge of 0.1% for each purchase or sale, amounting to around 2.7% annually, and the returns shown in the figure are net (though taxable).

Figure 1 - Value of $1 invested in buy-and-hold SPY and active, sentiment based strategy

If the result shown in Figure 1 appears too good to be true, it probably is. In particular, the blue line does not strictly represent out of sample performance (see below) and therefore perhaps it is closer to an upper bound on what the strategy could achieve. Nevertheless, I believe the potential for superior results is real.


Sector coverage is shown in Table 1. The core consists of State Street's nine S&P 500 Select Sector ETFs, which date from 1998. Unfortunately, attempting to expand beyond these to other asset classes such as fixed income, real estate, commodities, international equity, etc. quickly runs into data constraints, as many assets one would like to use have only been in existence a few years. Nevertheless, I've added three more ETFs to the coverage: IYR (US real estate, inception 2000); and LQD and SHY (fixed income, inception 7/31/2002). The full data set as shown in Table 1 is available with 10 years of history.

Table 1 - Sectoral coverage, 2002 - 2012

A Sentiment-Based Sector Rotation Strategy

Moving beyond the earlier contrarian strategy entails two steps: (1) partitioning the bullish sentiment index into more than two intervals, one of which represents extreme pessimism; and (2) formulating an asset allocation rule based on observed bullish sentiment.

  1. Partitioning bullish sentiment: "A Contrarian Guide to Market Timing," found strong evidence of above average (contrarian) returns during six week periods after observing bullish sentiment < 25%. In order to preserve this insight for the new strategy, I'll label as "bullish sentiment < 25%" any week beginning with an observation of bullish sentiment < 25%, together with the next five weeks regardless of how sentiment evolves during that time. These six week blocks of time constitute around 20% of the period 2002-12. For the remaining 80%, I divide bullish sentiment into 5 percentage point intervals: 25-30%, 30-35%, etc. and classify individual weeks according to the value of bullish sentiment in that week only. The relative frequencies of the different intervals over the period Aug. 2002 - Dec. 2011 are shown in the final row of Table 2. Bullish sentiment has a mean of 41 over the period and a standard deviation of 10.
  2. Mapping bullish sentiment into sector allocations: An obvious first approach to try is purely empirical -- selecting one or more sectors each week based solely on past performance. Table 2 shows average weekly returns for each sectoral ETF in each range of bullish sentiment. For instance, XLB returned an average of 0.74% per week during the 19.5% of weeks of bullish sentiment< 25%, .07% per week during the 4.9% of weeks of bullish sentiment in the range of 25-30%, etc. The top 3 sectors in each column/range of bullish sentiment are shaded in green. The table is based on the period Aug. 2002 - Dec. 2011, leaving Jan. - July 2012 for out of sample testing.

Table 2 - Weekly returns (%) for sector ETFs, conditional on ranges of bullish sentiment,2002-11

Since the best performing sectors in each column outpace SPY by 0.3-0.5% per week, it is easy to choose an allocation rule that generates very good in-sample results, with returns of 30% per year or more. But, identifying future top performers from past data is more difficult because sectoral returns exhibit strong mean reversion, both absolutely and relative to SPY. In other words, past performance is no indication of future results. Experimenting with different specifications and estimation periods failed to come up with any time series estimates that consistently improved on the cap weighted market average, SPY. Thus, for instance, selecting the best 1, 2, or 3 sectors from Table 2 gave significantly worse results out of sample (Jan.-July 2012) than buy-and-hold SPY. Likewise, using a rolling window of anywhere from 1 week to 5 years to update Table 2 on a weekly basis did a poor job of selecting assets over the period 2002-12. In brief, it is well known that short term price movements are essentially unpredictable and slicing up the data according to ranges of bullish sentiment does nothing to get around that.

A different approach is to link investor sentiment to sector performance independently using a structural framework. An initial, somewhat impressionistic attempt produced the encouraging result shown in Figure 1. The allocation rule uses the following simple logic (with sector selections shown in parentheses): (1) Bullish sentiment < 25% is likely to be accompanied by falling interest rates and rising consumer confidence (over the six week period), which should tend to favor interest sensitive and discretionary consumer sectors (IYR, XLY). (2) For moderately pessimistic bullish sentiment in the range 25-40%, generally poor market outcomes and heightened downside risks favor a cautious selection . (3) Optimistic bullish sentiment, above 40%, is more likely to occur during sustained expansions which should benefit cyclical sectors such as industry, raw materials and energy (XLB, XLE, XLI). Table 3 summarizes the allocation rule.

Table 3 - Asset allocation rule

This allocation rule generates the blue line plotted in Figure 1. As noted above, it is important to acknowledge that the model is not truly independent of the data because it was partly influenced by having seen Table 1. Thus, its performance over history can't be considered an independent statistical test. In particular, the cautious selection for bullish sentiment in the range 25-40% reflects the 2007-09 downturn, and this period explains a substantial part of the overall superior outcome. I'll return to this below. Nevertheless, the rule intentionally streamlines and smoothes asset allocation across the range of bullish sentiment, to avoid overfitting to the particular data in the table. A Bayesian approach, blending a structural prior (Table 3) with a sample-based likelihood (Table 2) offers a more robust methodology, but so far I've not tried to formalize such an approach.

Figure 2 plots performance over the period Jan.-July 2012 which more closely approximates an out of sample test. The strategy underperforms buy-and-hold SPY, especially during March and April, though that need not invalidate the approach since by random chance even a very good strategy will underperform its benchmark some of the time. It would require more data to test the strategy adequately. Table 4 below compares a few performance metrics for the active sentiment-based and buy-and-hold strategies. The former has a significantly higher expected return and lower risk, resulting in a much higher Sharpe ratio. However, it underperforms buy-and-hold SPY nearly one week out of two, so the results shown in Figure 2 are well within normal sampling variation.

Figure 2 - Active sentiment based and buy-and-hold SPY portfolios, Jan - July 2012

Table 4 - Performance metrics for the active sentiment based, contrarian, and buy-and-hold-SPY portfolios, 2005-12

Finally, the active, sentiment based strategy owes much of its superior performance to a sharp divergence from buy-and-hold SPY on a few occasions, especially during the bear market of 2007-08. A closer look at this period helps to explain how the sentiment-based portfolio outperforms. Table 5 shows the distribution and performance of portfolio holdings over the period June 2007-Feb. 2009. The active sentiment-based portfolio gains a cumulative 39%, while buy-and-hold SPY loses nearly 50%. The active, sentiment based portfolio outperforms across all ranges of bullish sentiment. However, nearly two-thirds of the cumulative outperformance occurs when sentiment is moderately pessimistic (25-40%) and SPY loses an average of 2.1% per week while the active sentiment based portfolio, parked in short term treasuries essentially treads water. Another nearly 30% of the divergence is attributable to (contrarian) investments in XLY and IYR during periods of extreme pessimism (< 25%). Only around 7% occurs during above average bullish sentiment (> 40%). Assuming that events such as the extreme bear market of 2007-09 will remain rare, Figure 1 likely exaggerates the outperformance that should be expected under more normal conditions. Omitting June 2007- Feb. 2009, the active sentiment based portfolio continues to outperform, but by a less awe-inspiring margin of 23% annually compared to 18% for buy-and-hold SPY.

Table 5 - Comparative performance of the Active Sentiment Based and buy-and-hold SPY portfolios in June 2007-Feb. 2009


Provisional results indicate that AAII bullish sentiment may have more to offer investors than an occasional contrarian buy signal. Because sectoral returns vary substantially at any level of bullish sentiment, asset selection potentially offers considerable rewards. The article demonstrates a sector rotation strategy that would have outperformed buy-and-hold SPY by a significant margin, especially in down markets. A methodology combining both structural and data driven elements would be better able to adapt to evolving market conditions.

Disclosure: I am long SPY, IYR, SHY, XLY. I wrote this article myself, and it expresses my own opinions. I am not receiving compensation for it (other than from Seeking Alpha). I have no business relationship with any company whose stock is mentioned in this article.