The Confidence Heuristic

In Moore’s experiment, volunteers were given cash for correctly guessing the weight of people from their photographs. In each of the eight rounds of the study, the guessers bought advice from one of four other volunteers. The guessers could see in advance how confident each of these advisers was (see table), but not which weights they had opted for.

From the start, the more confident advisers found more buyers for their advice, and this caused the advisers to give answers that were more and more precise as the game progressed. This escalation in precision disappeared when guessers simply had to choose whether or not to buy the advice of a single adviser. In the later rounds, guessers tended to avoid advisers who had been wrong previously, but this effect was more than outweighed by the bias towards confidence. …

Moore said that following the advice of the most confident person often makes sense, as there is evidence that precision and expertise do tend to go hand in hand. For example, people give a narrower range of answers when asked about subjects with which they are more familiar.

There are times, however, when this link breaks down. With complex but politicised subjects such as global warming, for example, scientific experts who stress uncertainties lose out to activists or lobbyists with a more emphatic message.

That is from New Scientist back in June.  So the key hard question is: when does confidence credibly signal expertise, and when it is just empty cheap talk?  Hat tip to Toby Ord.

GD Star Rating
Tagged as:
Trackback URL:
  • Matt

    What if every time an expert gives an answer of more than mildly confident, they have to make a bet that they are right. The amount of the bet would have to be proportionate to how confident they are and the seriousness of the claim.

  • michael webster

    Robin, your link to the New Scientist is wrong, here is the correct link:

    • Robin Hanson


  • michael webster

    “Experts are good at reporting relatively
    narrow intervals centered on true values, but
    they are no better than novices at reporting well calibrated,
    high-confidence intervals.

    Making good decisions
    about whether to consult experts in this
    context requires understanding what experts are, and
    are not, likely to deliver.”


  • kinbote

    whenever the nature of the domain of expertise precludes direct and unambiguous feedback of successful and unsuccessful predictions we should ignore the confidence of “experts”. confidence is a ”feeling” that is reliable only when we have no room to rationalize away failure.

  • billswift

    “The majority is never right”; when the majority tells you that if you walk off a cliff, you will go splat, they are probably right; when they tell you God exists and it is evil to even question it, they are wrong; the difference is immediate experience vs stories.

    As for the global warming example, everyone, especially those interested in OB or LW, should read The Deniers. The author still believes in agw even after assembling all the evidence in the book. Every one of the experts still accepts global warming, they apparently believe that only the information they are personally and professionally acquainted with was misused by the zealots and everything else was accurately reported by the media.

    • Andrew Kemendo

      To use Robin’s favorite term – the belief in agw is a signal to other scientists that they are competent AND compassionate. It doesn’t matter that the evidence does not match what they demand in other hypotheses.

  • Pzeffan

    In order to percieve any form of communication as a signal for expert advice, you must first have the means in place to interpret it as such. As far as assembling a toolkit to aid one in such a task , that would require at minimum an objective open mind that is experienced in critical thinking, well versed in the english language, and capable of weighing the likelyhood of the suggested outcome versus other competing scenarios.

    Secondly, it would help if the seeker was skilled in recognizing subtle behavioral cues in the self professed experts body language. The best way to lay the groundwork here would be to get into the “field” and work toward becoming an expert at something…anything…and then sharing that knowledge with others. In other words, it takes one to know one.

    We must become an architect of mind by accepting resonsibility for our reasoning, and design a way to recognize flaws in logic. Without utilizing the neccessary “software” needed to translate “code” into meaningful context, the uneducated and less confident turn to the advocates that choose to answer the call on thier behalf. We need to become confident seekers instead.

  • Pingback: Teflon colleagues « Flip Chart Fairy Tales

  • Pingback: The Confidence Heuristic » Dig for Leadership - Stories that try to make the world a better place.

  • Pingback: Weekly Roundup 41: A Curated Linkfest For The Smartest People On The Web | SimoleonSense