How did the markets do?


**edit** hint: there are only 2 crossings of reality and prediction.**edit 2** my data was on log scale. Near-term (for about a year) the futures market was roughly accurate...then they started diverging...inflation?

https://quitter.se/notice/1... opposite effect occurs: prediction vastly overshot real cost

Expand full comment

A very confusing conversation. Giving Perry E. Metzger the benefit of doubt perhaps he and Hal Finney were talking about two separate things. "Volatility" to the average uninformed person like me could either mean the change in the price of a commodity from now until the settlement time, or it could mean uncertainty in the future price. Without having read the followup entry, I think there are several conclusions one could have made from this discussion:

1) The original post unconsciously assumed that oil spot prices will continuously increase, at whatever rate, once oil production peaks, in other words that other factors would not affect that upward trend2) The expected spot price of a commodity in the future is affected by exactly the same factors as the price of a futures contract for that commodity, so they are the same thing3) Various complicated things affect the prices of relatively low-priced metals that are not meant to be stored in bulk, such as aluminium, which may cause downward trends contrary to interest rates/inflation4) Highly valued commodities like gold will have upward-trending futures as they have relatively low storage cost so are not sensitive to transient supply or demand issues around the world5) the downward bend in oil was due either to an initial partial payment by the buyer (and thus loss of interest), or an expected decrease in demand due to alternative sources of energy, or (most likely) to a reduction of supply issues that led to the predicted spike; but was most definitely not due to any "long-term storage costs" which only affect the linearity of a futures market with respect to its current price. For a commodity with low storage costs future demand will cause a large shift in current price which reduces the future spikyness, while for high storage prices it is smoothed mostly by changes in demand or supply which leads to more spikyness such as in natural gas.

Expand full comment

you are lost in a world of your own analysis:

"Back to oil. The strip tells us that there is more supply than demand - the market effectively pays the producers to not bring it to market by the amount of the spread differential. However, why does it then do inverted from the contango? The market "thinks" that there will be a time when demand will outstrip supply and thus, it will pay the producers to bring the oil to the market rather than leave in storage."

All meaningless. While you are fiddling Rome is burning. Bigger things are happening. Don't look at the facts . Everything will be okay. 100 dollars a barrel and above. Want me to predict a NLT date?

I don't need a big book of formulas to prove my hypothesis.

But it's not the end just a major transition in the evolution of civilization

Expand full comment

No offense, but that looks like a lot of "trees" to me. That is, you are analyzing the situation by studying detailed facts. My goal is to step back and look at the big picture without getting distracted by the facts(!) - to, as you say, see the "forest".

Expand full comment

Look at the forest not the trees:

1.everyone is having to go much deeper for oil.

2.The gulf state nations may really have reasons for wanting nuclear power other then dropping the bomb on Israel.

3. Demand for oil will increase with the further industrialization of china/india etc.

4. Why go after the Anwar to drill for oil if we don't have the need. Please don't talk about our dependence on foreign oil as the reason as what would a 10pct increase home grown oil do to our purchasing from the Saudi's? (Think about it)

5. Within how many years will it be before there are mass shifts in population due to the effect of rising seas. (Don't believe me:look at the ice caps and glaciers) Oh but that's not Global warming? Look at the ice caps and glaciers (Who cares what caused it: Look at the ice caps and glaciers!!! This will create a further capital drain, perhaps mass projects to save florida, the netherlands, california coastline, etc.(Don't believe a word I say - just keep watching.) Seeing is believing.

6. WAR: As resources become scarce as we begin the shift away from Fossil fuels man, we are already gearing up to make sure we own all the oil that's left.

7. We are building a wall around our Country to keep out the riff raff. If it gets bad here imagine how bad it will get elsewhere.

8. Our national guard and (unbelievable) state guard are being called up to their third extended tours in Iraq. That is the National guard - remember - they used to be weekend warriors. Go to Iran? Without a draft?

9. China is training pidgeons to fly under electronic direction. How bout that bird flue? China has shot a sattallite out of the sky. China is in the midst of a military buildup that we don't talk about. Russia, is not so benign anymore and is starting to grimice at its neighbours.

10. And drum roll please - The Vice president and President - I am sure - outed an undercover CIA agent who's husband was blowing the whistle regarding the real intelligence on the war in Iraq.

Well that about wraps it up - is anyone awake out there?

Oil at $100 plus- coming soon

Expand full comment

Hal - re your Dec 24 post

"One thing I've never seen explained is why the oil curve has the shape it does: up and then down. Has it always had that shape? "

No. It hasn't. A great deal of what you're seeing is a result of spreads calling the commodity to the marketplace - or paying the producer to keep it off the market. (Yes, there are many, many other factors - the answer below is simplified)

I'm a grain hedger, so I'll explain with grains. The spread between Jan beans and March beans closed at $.15 yesterday. This means, to me in my position, that the market will pay me $.075 per month to hold my grain from the cash market. Essentially, the market is paying me to keep grain in storage - the deferred price is higher and the strip slopes up.

However, when the commodity is in demand in the market place, an inverse occurs - the front price is ahead of the deferred price and it costs the producer/hedger to hold onto the commodity. If Jan beans were $.15 more expensive than March bean, then it'd cost me $.075 per month to hold the beans in my bins. Now - there may be reasons for me to do so, however, the market is clearly telling me to bring the bean to the cash pipeline now.

Back to oil. The strip tells us that there is more supply than demand - the market effectively pays the producers to not bring it to market by the amount of the spread differential. However, why does it then do inverted from the contango? The market "thinks" that there will be a time when demand will outstrip supply and thus, it will pay the producers to bring the oil to the market rather than leave in storage.

One additional factor - the effect of the "long only" commodity funds. Big investment pools that buy commodities as an "investment". I know they're always long, the locals know they're long and we take advantage of their rolling by front running the spread driving it out before they do. This adds an additional contango factor and whole additional level of complexity - and interest.

Expand full comment

I was intrigued by Peak Oil as an idea 18 months to 2 years ago. It seemed such a strange return to the fuel crises of my youth. I started following the energy news (and the more reasonable of the blogs) more closely, read a few (of the more reasonable) books, and came to broadly accept something like Peak Oil. I could say I fuzzily expect a peak, or something fuzzily like a peak.

What I think sources like Overcoming Bias teach me though is that I can't really say much about what that peak will look like. I certainly can't saw what the world after the peak will look like. There are way too many questions between here and there. We don't know which technologies can be improved, and we don't know what social trends will predominate.

Anyone who describes a post-peak world is layering a considerable set of assumptions in order to get there. As an example, how many optimists assume that energy will be cheap and plentiful after peak oil? How did they get there? Is there a rational path to that (or any other) outcome? Or is it an assumption that the universe owes us cheap energy?

This is an interesting story, and the continuing human reaction as it plays out is not a small part of the drama.

Expand full comment

Perry, what you say makes sense and I agree with it. However at the same time I think the point I made is valid as well, that there is an arbitrage opportunity if futures market prices are out of line with market expectations of future spot prices. I believe these two points of view are compatible, as I argued above. And as the Greenspan quotes that John D found illustrate, the shape of the futures strip does contain information beyond what is found in current spot prices. Back in the 90s it apparently had a different shape from today, and the question is what this tell us about the future.

I found an article that discusses these points in some detail, and includes a graph comparing the shape of the curves for different commodities:


See Chart C in particular which illustrates an oil price curve similar to the one I showed above, along with gold and silver, and non-monetary metals aluminum and copper. The gold and silver go up in a straight line for the reasons you have explained, but aluminum and copper both fall, again similarly to the aluminum curve I made today.

They do make an interesting point for why oil futures prices in particular might overestimate spot prices. The idea is that oil speculators with long positions are getting insurance, since if oil prices rise it will hurt the economy and could reduce speculators' income. By taking a long position they offset that risk, while short speculators take on more risk. This puts upward pressure on the futures market price and might cause it to be higher than expected spot prices.

The specific example they give shows a Reuters survey of economists predicting under $45 in 2010, compared to the oil futures price of over $60. Maybe I spend too much time with Peak Oilers but that $45 prediction sure looks low to me, and it's hard to credit it as superior to the market price of $60+.

Expand full comment


As I explained earlier, because you can store a commodity, the price for the futures contract tracks the current spot price modulo the interest rate and storage costs, not the future price. You can learn about this in most books on futures contracts. To someone with an economics background, this sort of argument should be straightforward. Naturally, the current best guess about the future price does alter the current price, but it tells you nothing about the beliefs about expected range.

Were my statement untrue, then there would be an amazing opportunity for arbitrageurs -- they could simply buy vast lots of commodities at spot, and lock in profits in the futures market with virtually no risk. You can choose not to believe me if you like, but then I'm afraid you would have to explain why it is that people could not arbitrage the price on a contract closing in two years against the current price, which is essentially what you would have to claim in order to claim that the price on the two year contract does not reflect the spot price. Indeed, were my statement untrue, why aren't you yourself out there making a fortune arbing the spot market against the futures?

As for the aluminum curve you posted, it looks rather strange to me -- the two completely linear domains are very odd, as is the denomination on the vertical scale.

Anyway, I've said as much as I think is productive on this topic.

Expand full comment

Perry, here is a curve I made today for Aluminum based on prices just before closing:


It looks pretty different from oil to me. First, oil (after the first month) rises then falls, while aluminum (after the first month) only falls. Second, if we restrict attention to the time frame up to the end of 2008, which is as far as the Aluminum futures go, oil rises to a peak during that time frame, while Aluminum falls. So during this time period the two commodities are going in opposite directions.

It is important to understand the degree to which futures prices reflect market anticipation of future spot prices. Certainly futures markets are more socially useful if they can be considered to apply this function, and in that case they offer great value to those of us seeking to Overcome Bias.

I have yet to see an explanation of how a futures contract price can substantially diverge from the market consensus future spot price. Clearly in that circumstance, the average trader can take a position with an expected profit, which should not be possible in an efficient market. Now there are costs to such a position: transaction costs, but they are not too large; foregone interest, but commodity contracts require only a small margin payment so this is small compared to other investments; and risk, but this applies equally to long and short speculators, and most markets have few hedgers, so it is not a major factor.

Expand full comment

Consistent with Hal Finney's treatment in this post ...

Here's how Alan Greenspan looked at crude oil futures at the beginning of an April 2004 speech: "The dramatic rise in six-year forward futures prices for crude oil and natural gas over the past few years has received relatively little attention for an economic event that can significantly affect the long-term path of the U.S. economy. Six years is a period long enough to seek, discover, drill, and lift oil and gas, and hence futures prices at that horizon can be viewed as effective long-term supply prices." (http://www.federalreserve.g... )

In October later that year Greenspan noted the earlier behavior of futures: "Between 1990 and 2000, although spot crude oil prices ranged between $11 and $40 per barrel for WTI crude, distant futures exhibited little variation around $20 per barrel. The presumption was that temporary increases in demand or shortfalls of supply would lead producers, with sufficient time to seek, discover, drill, and lift oil, or expand reservoir recovery from existing fields, to raise output by enough to eventually cause prices to fall back to the presumed long-term marginal cost of extracting oil. Even an increasingly inhospitable and costly exploratory environment--an environment that reflects more than a century of draining the more immediately accessible sources of crude oil--did not seem to weigh significantly on distant price prospects." (http://www.federalreserve.g... )

To quickly relate this discussion at least tenuously to the topic of bias (though bias here could be a risk premium consciously embedded into the market price)...

Futures as a biased predictor of spot in crude oil:

"This paper presents some empirical evidence on market efficiency and unbiasedness in the crude oil futures market and some related issues ... The evidence suggests that futures prices are neither unbiased nor efficient forecasters of spot prices" (http://www.sciencedirect.co... )

Expand full comment

I think it is important not to bias oneself against "peak oil" by assuming that such a prediction implies a Malthusian or Erlichian outcome.

Many minerals have, in fact, been exhausted over the millennia, but mankind has survived anyway, because we can come up with perfectly reasonable substitutes. For example, natural cryolite deposits more or less ran out decades ago, but we now use synthetic cryolite in aluminum smelting so the world hasn't really noticed.

Now, the "bias against Erlich" would hold that it is impossible that natural cryolite could have run out, but indeed it did. It just wasn't a catastrophe for the industry. The Erlichian view would have predicted catastrophe, but one must not "overcompensate" by assuming that resources cannot run out.

Indeed, I will point out that groups of humans have gone insane at times and destroyed the basis for their own survival -- see Easter Island, where the obsession with erecting monoliths lead to the elimination of all trees on the island.

The "peak oil" meme should be analyzed in two distinct parts. The first is, "is it credible that oil would eventually run out if extraction continues at current rates", and the answer is an obvious yes. When it would run out, I can't say, but we're clearly removing it faster than natural processes re-create it, and mathematics tells you what the ultimate destination of those curves is -- at some point, extraction of fossil oil will of necessity decline. The second part is "should we, on a policy level, care?", and there I think the answer is "no", because there are adequate substitutes, and the market can deal with finding them.

A possible third question is "how soon will we hit the peak on the production curve, because I want to know how to invest", and there I can't answer. Every entrepreneur must figure that out on their own.

Expand full comment

Hopefully this is relevent to the subject of bias - what interests me about the peak oil debate is how often the debate on future shortages of various commodities endlessly re-occurs. This started with Maltheus, but there are lots of other examples since (most recently the Limits to Growth report, Paul Erlich and the peak oil protagonists). The case is always presented very strongly; with little doubt as to the imminent terrible outcome should the chosen policy prescriptions not be followed.

Of course, so far, none of these terrible outcomes has happened, even with the proposed prescriptions not being followed. This is probably a good thing as the prescriptions generally were very expensive or required significant sacrifices. But the failure of these previous predictors does not seem to deter the current lot, even though their arguments seem to me to basically be the same.

Perhaps the attractiveness of the scenario of imminent commodity shortages is somehow related to a folk or even genetic memory of the days when life was largely agricultural and this tendency was a useful reminder to prepare for lean days even in a time of plenty.

Expand full comment

This forum is about Overcoming Bias (TM). Since the topic of peak oil is controversial, it is plausible that it might exemplify important biases or ways to correct bias. But I haven't seen that very clearly in this post or these many comments. It is good that we just don't talk abstractly about bias, but grapple with concrete examples. But let's try connect those concrete examples back to the subject of bias; I don't want this to just be a general forum for discussing controversial issues.

Expand full comment

Of course they are uneconomical now. We still burn hydrocarbons for energy so obviously it's not economical to convert electricity to hydrocarbons. However, many sources of energy in the future may be better for making electricity than for making hydrocarbons. If electricity was X times cheaper than oil, on a joules per dollar basis, we could economically convert it to oil. I want to know X. A good starting point is to ask the energetic efficiencies of electrolysis and of reduction of CO2.

Expand full comment

Right now, all methods of synthesizing long chain alkanes are uneconomical. However, that is unlikely to remain the case permanently. Biological systems efficiently synthesize long chain substituted alkanes, to whit, fatty acids, so in principle we should be able to ultimately harness those same synthetic pathways or find even better ones.

Expand full comment