Hindsight bias

Hindsight bias is when people who know the answer vastly overestimate its predictability or obviousness, compared to the estimates of subjects who must guess without advance knowledge.  Hindsight bias is sometimes called the I-knew-it-all-along effect.

Fischhoff and Beyth (1975) presented students with historical accounts of unfamiliar incidents, such as a conflict between the Gurkhas and the British in 1814.  Given the account as background knowledge, five groups of students were asked what they would have predicted as the probability for each of four outcomes: British victory, Gurkha victory, stalemate with a peace settlement, or stalemate with no peace settlement.  Four experimental groups were respectively told that these four outcomes were the historical outcome.  The fifth, control group was not told any historical outcome.  In every case, a group told an outcome assigned substantially higher probability to that outcome, than did any other group or the control group.

Hindsight bias matters in legal cases, where a judge or jury must determine whether a defendant was legally negligent in failing to foresee a hazard (Sanchiro 2003). In an experiment based on an actual legal case, Kamin and Rachlinski (1995) asked two groups to estimate the probability of flood damage caused by blockage of a city-owned drawbridge. The control group was told only the background information known to the city when it decided not to hire a bridge watcher. The experimental group was given this information, plus the fact that a flood had actually occurred. Instructions stated the city was negligent if the foreseeable probability of flooding was greater than 10%. 76% of the control group concluded the flood was so unlikely that no precautions were necessary; 57% of the experimental group concluded the flood was so likely that failure to take precautions was legally negligent. A third experimental group was told the outcome andalso explicitly instructed to avoid hindsight bias, which made no difference: 56% concluded the city was legally negligent.

Viewing history through the lens of hindsight, we vastly underestimate the cost of effective safety precautions.  In 1986, the Challenger exploded for reasons traced to an O-ring losing flexibility at low temperature.  There were warning signs of a problem with the O-rings.  But preventing the Challenger disaster would have required, not attending to the problem with the O-rings, but attending to every warning sign which seemed as severe as the O-ring problem, without benefit of hindsight.  It could have been done, but it would have required a general policy much more expensive than just fixing the O-Rings.

Shortly after September 11th 2001, I thought to myself, and now someone will turn up minor intelligence warnings of something-or-other, and then the hindsight will begin.  Yes, I'm sure they had some minor warnings of an al Qaeda plot, but they probably also had minor warnings of mafia activity, nuclear material for sale, and an invasion from Mars.

Because we don't see the cost of a general policy, we learn overly specific lessons.  After September 11th, the FAA prohibited box-cutters on airplanes - as if the problem had been the failure to take this particular "obvious" precaution.  We don't learn the general lesson: the cost of effective caution is very high because you must attend to problems that are not as obvious now as past problems seem in hindsight.

The test of a model is how much probability it assigns to the observed outcome.  Hindsight bias systematically distorts this test; we think our model assigned much more probability than it actually did.  Instructing the jury doesn't help.  You have to write down your predictions in advance.  Or as Fischhoff (1982) put it:

When we attempt to understand past events, we implicitly test the hypotheses or rules we use both to interpret and to anticipate the world around us. If, in hindsight, we systematically underestimate the surprises that the past held and holds for us, we are subjecting those hypotheses to inordinately weak tests and, presumably, finding little reason to change them.

---

Fischhoff, B. 1982. For those condemned to study the past: Heuristics and biases in hindsight. In Kahneman et. al. 1982: 332–351.

Fischhoff, B., and Beyth, R. 1975. I knew it would happen: Remembered probabilities of once-future things. Organizational Behavior and Human Performance, 13: 1-16.

Kamin, K. and Rachlinski, J. 1995. Ex Post ≠ Ex Ante: Determining Liability in Hindsight. Law and Human Behavior, 19(1): 89-104.

Sanchiro, C. 2003. Finding Error. Mich. St. L. Rev. 1189.

Comments

[Robin Hanson]

So the obvious solution is to write down forecasts in advance. And of course in the particular cases where hindsight bias is larger, this will produce a large benefit. But some might worry about hindsight bias in recommending advance forecasts, as it is not so easy to tell ahead of time which situations will have the worst hindsight bias. How can we get an unbiased estimate of the value of overcoming hindsight bias with advance forecasts?

[Nathan Iver O'Sullivan]

Chapter 11 of the 9/11 commission's report, available here, shows the commission was very wary of hindsight bias. The failure to prevent the attacks is said to represent a "failure of imagination," meaning the intelligence community used the wrong model in evaluating terrorist threats.

[Aaron Luchko]

This made me think of a specific instance of hindsight bias that always annoys me. Consider any game of chance where at some point the person is given the choice of whether to make a wager or not.

Once they see how the wager would have turned out one is almost guaranteed that if the wager would have won they'll say to make the wager would be the right decision and if the wager would have lost vice-versa. This holds even if they were already aware of the odds before hand.

[pdf23ds]

Eliezer, I'm curious as to what you think of Feynman's take on the Challenger disaster. Do you think he was succumbed to hindsight bias in his judgments or recommendations?

[Eliezer Yudkowsky]

It appears to me that Feynman did his best to talk about a general policy that would have been required to prevent all problems of the same level as seen without benefit of hindsight, rather than saying "Why didn't you fix the O-Rings, you idiots?" or setting up a Low Temperature Testing Board.

[Andrew]

Eliezer,

You write: "I'm sure they had some minor warnings of an al Qaeda plot, but they probably also had minor warnings of mafia activity, nuclear material for sale, and an invasion from Mars." I doubt they had credible warnings about an invasion from Mars. But, yeah, I'd like the FBI etc. to do their best to stop Al Quaeda plots, Mafia activity, and nuclear material for sale. I wonder if you're succumbing to a "bias-correction bias" where, because something _could_ be explainable by a bias, you assume it _is_. Groups of people do make mistakes, some of which could have been anticipated with better organization and planning. I have essentially no knowledge of the U.S. intelligence system, but I wouldn't let them off the hook just because a criticism could be simply hindsight bias. Sometimes hindsight is valid, right?

[Eliezer Yudkowsky]

The notion being that following up on all warnings of equal then-apparent severity, without benefit of hindsight, would have been a prohibitively expensive general policy. Especially since you would not have any information about "terrorism" being the pet problem of the '00s, rather than, say, an unpaid military officer launching a Russian ICBM, runaway greenhouse warming, a home biologist making superviruses, asteroids, unFriendly AI, etc.

It's all very well to talk about mistakes that could have been anticipated, yet somehow, they don't seem to be anticipated.

[David J. Balan]

Of course it's always hard to know what truth is in situations like this, but there appears to be evidence that the people who were actually in charge of preventing terrorism were actively worried about something much like what actually happened, and were ignored by their superiors.

[Eliezer Yudkowsky]

David, which other 50 things were they actively worried about?

As Fischoff (1982, above) writes:

Any propensity to look no further is encouraged by the norm of reporting history as a good story, with all the relevant details neatly accounted for and the uncertainty surrounding the event prior to its consummation summarily buried, along with any confusion the author may have felt (Gallie, 1964; Nowell-Smith, 1970). Just one of the secrets to doing this is revealed by Tawney (1961): "Historians give an appearance of inevitability to an existing order by dragging into prominence the forces which have triumphed and thrusting into the background those which they have swallowed up" (p. 177).'

Gallie, W. B. Philosophy and the historical understanding. London: Chatto & Windus, 1964.

Nowell-Smith, P. H. Historical explanation. In H. E. Keifer & M. K. Munitz (Eds.), Mind, science and history. Albany, N. Y.: State University of New York Press, 1970.

Tawney, R. H. The agrarian problems in the sixteenth century. New York: Franklin, 1961.