Bias Toward Certainty

Before the Iraq invasion, President Bush did not say, "I think that there is a 60 percent chance that Saddam has an active WMD program."

Al Gore does not say, "I think there is a 2 percent chance that if we do nothing there will be an environmental catastrophe that will end life as we know it."

Instead, they speak in the language of certainty.  I assume that as political leaders they know a lot better than I do how to speak to the general population.  So I infer that, relative to me, the public has a bias toward certainty.

Another piece of evidence of that is an anecdote I cite about my (former) doctor.  Several years ago, he said I needed a test.  I did some research and some Bayes’ Theorem calculations, and I faxed him a note saying that I did not think the test was worth it.  He became irate.

I think that one reason that our health care system works the way it does is that it does not occur to anyone to say, "OK, I can live with that level of uncertainty."  Instead, we must have the MRI, or the CT scan, or whatever.  Even, as in my case, when the patient is willing to live with uncertainty, the doctor has a problem with it.

Another way that bias toward certainty shows up is in the way we handle disagreement.  People don’t say that there were differences within the intelligence community about the probability distribution for Saddam having WMD.  They say that Bush manipulated the intelligence.  And they are right, in the sense that he tried to make it sound certain. 

My point is that what comes naturally to a lot of people on this blog–thinking in Bayesian terms–is in fact very unnatural in general.  It seems as though outside of the realm of sport betting, people don’t like to think in terms of chance.  Maybe there are realms where even those of us who are more Bayesian than most are victims of bias toward certainty.

GD Star Rating
Tagged as:
Trackback URL:
  • Although it I am digressing slightly from your main point, I think it bizarre that (AFAIK) pre-med programs require calculus rather than basic stats. A Tufte-Tukey based exploratory data analysis course would help encourage statistical thinking, and an intro to stats course could further develop the ability to think in terms of odds, probabilities, etc.

    However, although that would help the doctor in dealing with you, doctors in particular need to project certainty. There is research evidence to suggest that the doctor’s confidence in the efficacy of the treatment they are advocating improves its efficacy. Apparently, if the doctor doesn’t believe strongly in his treatment we won’t respond as fully. Curious, no?, and consistent with your hunch that we generally want certainty.

    Other evidence consistent with a desire for certainty is the demand for forecasts by experts despite the accumulating evidence for the inability of experts to outforcast the well-informed lay observer. (On this, see Philip Tetlock’s recent book: Expert Political Judgment: How Good Is It? How Can We Know?)

  • josh

    Rather than saying, people don’t think like Bayesians, isn’t it more likely that if you were trying to influence somebody else’s Bayesian analysis to be closer to your own, you should try to project a level of certainty that overshoots as they will discount your oppinion somewhat. Perhaps this game is one of the reason’s that people discount other peoples opinions in the first place. Also, both Bush and the Doctor were selling something, not necessarily trying to convince you that they were correct.

  • I had a similar experience with a doctor who was pushing beta blockers. I have high cholesteral and he, without asking, wrote me a prescirption for lexipro or something. I asked why and he told me that it’d reduce my chance fo heart-attack by 40%. I asked what my chance of heart attack currently stood at? 50%? 70%? He didn’t know, and wouldn’t venture a guess. He had simply arrived at the certainty that a reduction of probablity, without knowing the probability istelf, meant the decision was a no-brainer.

    He was also irate when I declined to take the drug.

  • You are basically correct about the WMD issue. However, I cannot resist pointing out that the WMD over which there was actual disagreement were chemical weapons, with the possibility of a bioweapons lab. However, it was fully agreed upon that Saddam had no delivery capability for chemical weapons. Therefore, although there was maybe a slight case over bio weapons (and whoever it was who sent that anthrax back in 2001 still has neither been identified nor caught, not that that amounted to much), there was no reason whatsoever to be concerned about whatever chemical weapons Saddam might have had as an actual threat to US national security. Even if they had existed, there was no justification at all for invading Iraq to go after them.

  • TGGP

    Josh, shouldn’t people then assume that people claiming certainty are exaggerating, and discount those statements more than those of people who claim a high but not 100% certainty?

  • Matthew

    shouldn’t people then assume that people claiming certainty are exaggerating, and discount those statements more than those of people who claim a high but not 100% certainty?

    I do this, both when I feel their opinion is probably true and when I judge it probably incorrect.

    True believers and true disbelievers are equally impermeable to new information.

  • TGGP and Matthew are right – a constant and known degree of overconfidence should not interfere with information transmission – the problem is variation in overconfidence.

  • FWIW, in one sector of climate science it has recently become very fashionable to adopt a position of maximal uncertainty. This has the politically convenient effect of assigning high probability to a future climate-change-induced disaster and thus any reasonably risk-averse cost-benefit analysis supports the taking of strong action.

    It’s a short step from the vacuous but emotive “we can’t rule it out” to assigning a 5-10% probability…

  • Douglas Knight

    Barkley Rosser:
    I put the odds at 20% that the person responsible for the anthrax has been identified, caught, and released.

  • JMG3Y

    With regard to medicine, the evidence-based medicine movement is an attempt to improve physician decision making. Emerging as a term in the early 90’s, Googling “evidence-based medicine” yeilds about 1.2 million hits. With regard to calculus, statistics is probably more useful as a pre-med requirement but on the other hand one typical undergrad biostats class likely doesn’t make one a Bayesian. To improve clinical practice, specific methods are being developed to summarize the best evidence for physicians. Google “Cochrane Collaboration” to find the mother lode of these efforts.

    “Respectful Insolence” is a blogging academic surgeon who covers this topic. Time magazine has an article on EBM under the title “Are Doctors Just Playing Hunches?”, Thursday, Feb. 15, 2007 By CHRISTINE GORMAN

    And yes, they often are. Medicine involves tinkering with very complex biological systems that have many idiosyncracies.

  • billswift

    “Question Authority” – I haven’t found the original source.

    “Don’t pamper yourself, question your own beliefs as relentlessly as you question authority.” – William Swift

    I have found the best way to keep from falling for any sort of false information is to be skeptical, and to be more skeptical when I would prefer it to be true. Always be most skeptical of what you most want to be true.

  • Steven Harbour

    I wonder if there is not a social biology expiation for the tendency of speakers to overstate their certainty. One could argue that in a small hunter gather tribe, in a crises situation, a decisive response presented the best chance for survival. Indeed, almost any decisive action, for example fight or flee, is preferable to indecision. Groups who followed leaders who were decisive had a greeter chance of survival. Leaders either did not see alternatives or believed that the verbalization of alternatives would be correlated with indecisiveness. As a result we are more “comfortable” with a leader (politician or doctor) who talks in terms of certainty, even when we understand intellectually that there is no certainty.