The New Generation of Diversified Commodity Indexes

Aug. 06, 2007 5:00 AM ETDBC, GSG, DJP2 Comments
Hard Assets Investor profile picture
Hard Assets Investor

As equity sectors have shown, heavy investor demand in an asset class can produce a quest for more innovative indexes. Now, with oil over $70 a barrel, gold over $650 an ounce, and corn silos holding the wealth of Iowa, the race is on to develop new commodity indexes that can add basis points of performance octane.

In equities, investors still debate: 1) whether passive indexing produces superior performance vs. active managers; and 2) whether new indexing techniques (e.g., fundamental weighting) can add value vs. the tried-and-true. In commodities, thanks to a depth of recent academic research, such debates are being put to rest.

It is now clear to most informed investors that: 1) diversified commodity indexes have a strong track record of outperforming spot commodity prices and even individual commodity futures contracts; and 2) innovations have the potential to add performance vs. traditional indexing, especially for investors with insight into commodity trends. This article delineates key differences between the first generation of diversified commodity indexes, which we will call “1.0,” and a new set of 2.0 challengers. It also summarizes the relevant scholarship upon which 2.0 innovations are built.

Diversified Commodity Indexes 1.0

Let’s start with the commodity equivalents of the Dow Industrials and S&P 500 indexes. The table below examines three of the most popular, traditional, diversified commodity futures indexes and their characteristics. There are others, but these are the biggies.

Generation 1.0 – Diversified Commodity Futures Indexes

Dow Jones-AIG Commodity Total Return Index

S&P GSCI Total Return Index

Reuters Jefferies CRB Index

(back-test history)

1998 (1970)

1998 (1991)


Index weighting

Liquidity and world production; 15% max. per contract

World production

Fixed weight to petroleum; other classes are weighted based on liquidity


3-month T bills

3-month T bills

3-month T bills

Component contracts




Rebalancing of weights




Rolling method

Roll of the front-month contract

Roll of the front-month contract

Roll of the front-month contract

Recent energy weighting***




Investable via


Mutual fund [PCRAX],


Mutual fund [QRAAX]


* In February of 2007, Standard & Poor’s acquired the GSCI from Goldman Sachs.
** Originally called the CRB Index, the name and methodology were substantially revised in 2005.
** Weights calculated by Ibbotson, as published 3/27/06 in Strategic Asset Allocation and Commodities.

The major difference among the three traditional indexes is the greater weight to the energy complex in GSCI – a result of its weighting method based solely on world production value. The use of “liquidity weights” by the other two indexes allocates greater index value to storable commodities, such as grains and precious metals

According to a study conducted by Ibbotson Associates for PIMCO, these indexes have delivered risk/return profiles comparable to U.S. stocks since their inception dates (and including back-test data), as summarized in the table below.

Analysis of Return and Risk Measures for 1.0 Indexes


Period Measured

Compound Annual Return

Standard Deviation


1/92 to 10/05




1/70 to 10/05




1/94 to 11/05



DJ Wilshire 5000

1/71 to 10/05



Source: Strategic Asset Allocation and Commodities, Ibbotson Associates commissioned for PIMCO; published March 27, 2006.

All three indexes also have produced low correlations with equities and bonds, as shown in the table below.

Correlations of 1.0 Commodity Indexes with Stocks and Bonds (1/92 to 5/05)

Commodity Index

Correlation with…

S&P 500 Index


Lehman Aggregate













Source: The Tactical and Strategic Value of Commodity Futures, Claude B. Erb and Campbell R. Harvey, January 12, 2006.

The Heavy Influence Of The “Big Five” Contracts

In decomposing long-term returns on futures indexes and individual futures contracts, researchers have found that spot commodity prices are not a powerful influence on long-term performance. A much greater influence has been the contributions of five futures contracts in which annualized historic returns have been far in excess of changes in spot prices.

Annualized Returns from 4/83 to 4/04

Futures Contract

Spot Price

Crude oil



Heating oil



Gasoline (since 1/85)






Live cattle



Source: Structural Sources of Return and Risk in Commodity Futures Investments, Hilary Till, April 2006.

What elixir helped these five contracts significantly outperform the corresponding spot commodities? The answer is their “term structures” and particularly their historic tendency toward “backwardation.” In simple language, this means that near-expiration futures contracts cost more than longer-term contracts; that oil today costs less than oil tomorrow.

Since the 1.0 indexes hold and roll over front-month contracts, backwardation allows them to earn small increments of “roll yield” profit systematically. Over time, the rolled-up profits in just five contracts have made 1.0 commodity indexes star performers. Currently, these five contracts compose 49% of the weight of the GSCI and a somewhat lesser portion of the other two 1.0 indexes.

The opposite of backwardation is “contango,” which means that near-term contracts cost less than longer-term. Contango creates a cost for futures indexes and a drag on performance. Research conducted by Daniel Nash and Boris Shrayer of Morgan Stanley confirmed a strong correlation between strong long-term futures performance and backwardation and weak long-term performance and contango. For the period 1983-2004, the five contracts shown in the table exhibited the strongest tendency to trade in backwardation. Research conducted by Barry Feldman and Hilary Till concluded that over five-year periods, backwardation/contango explains 64% of the variation in futures returns.

Why have some contracts historically traded in backwardation and others contango? Theorists say that backwardation implies a scarcity of a commodity in the spot market, so it tends to occur in commodities that are perishable and can’t easily be stored. However, this theory has limits in explaining historic relationships. For example, while most energy complex contracts have benefited from historic backwardation, natural gas has tended toward contango. Live cattle historically has trade in backwardation, while lean hogs have typically been contangoed. And hogs aren’t really harder to feed than cattle.

Researchers also believe that, over time, commodity cycles create structural changes in backwardation-contango trends. For example, Hillary Till suggests that the crude oil market may have recently shifted to “structural contango,” in which case crude future might not be as rewarding to indexes as in the past. Indeed, the energy markets have been in vicious contango during much of the 21st century, although that trend recently reversed. Erb and Harvey have written that “the historical pay-off from investing in commodity futures with above average term structure characteristics is an example of a feature of the data which may disappear in the future.” If this proves true, the 1.0 indexes could lose part of the performance mojo that has produced equity-like returns in the past.

The Diversification Benefit

Erb and Harvey have documented that diversified commodity indexes have an extra component of return not found in individual futures contracts – a “diversification benefit.” This return is achieved because: 1) many individual commodity contracts are volatile (high standard deviations); 2) the correlation among most pairs of commodity contracts is fairly low (averaging 0.09, according to their research); and 3) futures indexes periodically rebalance components. Erb and Harvey describe this component of return as “turning water into wine” – an advantage of well-diversified and rebalanced indexes that may be captured in commodities better than in any other asset class. Their models have shown that the diversification benefit can add an excess annual portfolio return of 3-4% per year compared to the average of component contracts. The more volatile and low-correlating components an index contains, and the more frequently it rebalances, the greater this benefit can be.

Generation 2.0 Indexes

The most interesting innovations in commodity indexes to date are found in recently introduced products launched by Deutsche Bank and a collaborative venture between UBS and Bloomberg. These new indexes offer opportunities to boost performance by: 1) adding flexibility to the index’s term structure; and 2) in the case of the UBS Bloomberg products, giving investors the ability to choose customized indexes that can benefit from tactical evaluation of term structure changes, while also potentially boosting the diversification benefit through the selection of volatile, low-correlating contracts.

Deutsche Bank Optimum Yield Indexes
In 2006, Deutsche Bank (DB) created a variant on its series of Liquid Commodities Indexes which it trademarked as “Optimum Yield [OY].” These indexes introduced a rules-based method for rolling expiring contracts so as to (in the words of DB) “maximize the positive roll yield in backwardated term structures or minimize the negative roll yield in contangoed markets.” On the first day of each month in which an index component is scheduled to expire, the OY formula performs an evaluation of similar contracts expiring in the following 13 months. It then chooses as a replacement the contract with the “highest implied roll yield.” DB believes that this method can be especially helpful in boosting yield if historical term structure relationships change – i.e., if crude oil shifts from backwardation to contango. The method also has the flexibility to take advantage of anomalies in term structures – i.e., one or two backwardated months in an otherwise contangoed term structure.

In back-testing the concept over the period 1988-2006, DB found that the OY roll formula added 52 basis points of annual return, compared to the same index with a traditional rolling method. The OY method also reduced back-tested index volatility from 18.93% to 15.32%. Currently, the OY concept is investable via a growing group of ETFs under the DB-PowerShares brand.

UBS Bloomberg Constant Maturity Commodity Index Family [CMCI]

In January of 2007, UBS AG and Bloomberg joined forces to introduce a novel concept in commodities indexing – a series of indexes on 28 individual commodity futures contracts and a Composite Index [CMCI] that combines them. Each index is available in multiple “tenors,” each of which reflects a constant maturity ranging from 3 months to 3 years. The CMCI includes all available sub-indexes and tenors, thus providing diversification not only across commodity contracts but along the length of the term structure; it holds them all. Sub-index component contracts are weighted similarly to the DJ-AIG, with 32% in energy, 29% in agriculture, 29% in industrial metals, 5% in precious metals, and 4% in livestock.

The “constant maturity” of each tenor is created by purchasing the two contracts that surround it in time on a time-weighted basis. This helps to average out term structure anomalies. Perhaps the most interesting feature of the CMCI indices is the potential to combine sub-indices into customized “vertical” or “horizontal” indexes. A vertical index includes all sub-indices available in a given constant maturity – e.g., the 1-year constant maturity for all 12 contracts that offer this tenor. A horizontal index includes all available constant maturities for a given sub-index – e.g., all seven tenors available for crude oil.

Comparing 2.0 Concepts

Deutsche Bank and UBS-Bloomberg have adopted conceptually different solutions for reducing the cost of contango, and either solution may outperform the 1.0 indexes if several of the “big five” contracts shift from structural backwardation to contango, as researchers have warned. The flexibility of the Deutsche Bank formula appears to have more ability to take advantage of irregular or rapidly changing term structures. The UBS-Bloomberg approach gives indexed investors more control in selecting the most favorable maturities on the term structure. (Typically, the cost of contango is greatest in rolling from the front-month to the next available month.)

A major competitive advantage of the UBS-Bloomberg concept may be the flexibility it offers to 1) customize index components that can benefit from backwardation trends; and 2) customize index components that can “turn water into wine” by maximizing the diversification benefit of rebalancing. (UBS-Bloomberg indexes rebalance to target weights monthly.) While commodity term structures can shift over time, they typically demonstrate momentum over the short term. Therefore, a savvy index investor may benefit by: 1) selecting those sub-index components of the CMCI that are currently in backwardation; and 2) selecting those sub-index components that have the highest volatility and the lowest correlations with each other (to maximize diversification benefit). No other commodity index currently on the market offers such pick-your-own-index flexibility.

One caveat should be mentioned, however. The UBS-Bloomberg indexes, designed primarily for institutional investors, initially were investable only via customized derivative transactions (swaps, options and structured notes). The recent growth in commodity index demand, however, has been in listed products – futures, options and ETFs. In commodity ETFs especially, the trend has been for first-movers to create sustained market share leadership that late-comers find difficult to overcome.

While Deutsche Bank-PowerShares has a first-mover advantage in implementing 2.0 commodity indexing concepts via ETFs, the DB indexes also may have a shortcoming – namely, they include relatively few components, which could reduce their ability to participate in backwardation or maximize diversification benefit. The flagship of the OY series, the DB Commodity Index Fund (DBC), participates in just six components: aluminum, corn, gold, heating oil, light crude and wheat.

[As this paper was being published, Goldman Sachs rolled out an exchange-traded note [ETN] tied to an enhanced version of the GSCI index. This new ETN follows a similar pattern as the DBC fund, but relies on an index with more components – albeit a very heavy weighting in energy. It offers yet another opportunity for investors to improve their commodity returns.]

One irony of all these new products is that they are hitting the market just as the recent period of contango has abated; in fact, the energy markets have ticked back over into backwardation. Still, if Till and others are right and the long-term trend is towards contango, these strategies could still come to the fore; moreover, there is no reason to say they won’t outperform or at least equal the 1.0 indexes, even in backwardated markets.

In summary, in perhaps no other area of modern finance have academic researchers pointed the way to enhanced index performance opportunities more than in commodities.

Source Papers

Structural Sources of Return and Risk in Commodity Futures Investments, Hilary Till, EDHEC Risk and Asset Management Research Center, April 2006.

The Strategic and Tactical Value of Commodity Futures, Claude Erb and Campbell Harvey, Financial Analysts Journal, March/April 2006.

Facts and Fantasies about Commodities Futures, Gary Gorton and K. Geert Rouwenhorst, Financial Analysts Journal, March/April 2006.

Strategic Asset Allocation and Commodities, Ibbotson Associates as commissioned by PIMCOI, March 27, 2006.

Separating the Wheat from the Chaff: Backwardation as the Long-Term Driver of Commity Futures Performance, Barry Feldman and Hilary Till, EDHEC Risk and Asset Management Research Center, 2006.

This article was written by

Hard Assets Investor profile picture
4.12K Followers ( is a Van Eck Associates-sponsored, research oriented Web site devoted to sharing ideas about hard assets investing. The site has been developed as an educational resource for both individual and institutional investors interested in learning more about commodity equities, commodity futures and gold (the three major components of the hard assets marketplace). The site focuses on hard assets investing, without endorsing or recommending any particular investment product or approach. Visit: Hard Assets Investor (

Recommended For You

Comments (2)

To ensure this doesn’t happen in the future, please enable Javascript and cookies in your browser.
Is this happening to you frequently? Please report it on our feedback forum.
If you have an ad-blocker enabled you may be blocked from proceeding. Please disable your ad-blocker and refresh.