To get back to the substance of the speculation debate, let me reprise something I've discussed for years to illustrate the vapidity of many of the arguments about speculation. I've been making this point for more than 6 years: it was a focus of a presentation I made at a conference in Champuloc, Italy in January 2008.
It relates to the use of position data to attempt to identify the price impact of speculation. You know the drill. Speculators are huge net long, so they must be driving up prices, right? We've seen that argument repeated over and over and over again.
But here's the thing. For every long, there's a short. So how do you know the net long speculators are driving prices up? Why aren't the net short hedgers driving down prices?
Here's the example I keep coming back to that illustrates the stupidity of using net speculative positions to claim that prices are too high or too low.
Back in the 1990s and early 00s, gold prices were low. Very low. $300 and below. Back then, the hue and cry was that prices were artificially low because...wait for it...producers were massively short because of hedging programs.
Well, if producers were massively short that means that speculators were massively long. So if speculators drive prices, why weren't gold prices stratospherically high in the late-90s early-00s? After all, supposedly in 2008, and the last couple of years, the massive long speculative positions were inflating prices. Why didn't the massive long speculative gold positions a decade ago inflate gold prices? Flipping things: If short commercial positions were depressing gold prices a decade ago, why didn't they depress oil prices in 2007-2008, and over the last couple of years?
Hence the danger of superficially examining net positions and claiming that one side of the market is inflating (or depressing) prices: an equally legitimate argument is that the other side of the argument is depressing (or inflating) prices.
But the point is that neither argument is legitimate: both are equally illegitimate. Derivatives positions net out to zero. Derivatives are in zero net supply. Looking at one side of the market, and ignoring the other, makes no sense.
Basic finance theory says that if anything, net positions should predict risk premia, and price trends. In a simple Keynes-Kaldor model, if speculators are net long, prices should drift up. Or not: if the price of the commodity in question is uncorrelated with the pricing kernel and markets are well-integrated, prices won't drift at all even if speculators have a net position.
But the basic point is that in standard finance speculation and hedging affect risk premia, not price levels. The relationship between positions and risk premia depends on the integration between the commodity market of interest and the broader financial markets, and the covariation between the commodity price and the pricing kernel. If the "financialization" of commodities means anything, it means that commodity markets have become more closely integrated with the broader financial markets, and that this has impacted risk premia. The evidence of Hamilton and Wu (and UH PhD student Bingxin Li) points exactly in this direction: risk premia in oil plunged to near zero in the mid-2000s, when index trading increased dramatically.
Any impact on the level of prices overall is due to indirect effects of risk premia: risk premia affect the costs of hedging which affect output and storage and consumption, which affect prices. But whatever those knock-on effects are, whether prices rise or fall, they are likely efficiency enhancing: they reflect the consequences of a more efficient allocation of risk. (By the way, I explained this to Chilton years ago. Talk about pearls before swine.) (I say "likely" because of the usual caveats about the difficulty of making welfare comparisons in incomplete markets.)
So word to the wise. If you want to make a persuasive argument about the impact of speculation (or hedging) on the level of spot prices (rather than on the difference between spot and futures prices), don't say anything about net derivatives positions. Because the market as a whole has no net position.