Disagreement Experiment

A lab experiment induces common priors, tells each person of the actions of others, and yet still finds disagreement, in conflict with predictions from common knowledge of rationality:

We look at choices in round 1, when individuals should still maintain common priors, being indifferent about the true state. Nonetheless, we see that about 20% of the sample erroneously disagrees and favors one point of view. Moreover, while other errors tend to diminish as the experiment progresses, the fraction making this type of error is nearly constant. One may interpret disagreement in this case as evidence of erroneous or nonrational choices.

Next, we look at the final round where information about disagreement is made public and, under common knowledge of rationality, should be sufficient to eliminate disagreement. Here we find that individuals weigh their own information more than twice that of the five others in their group. When we look separately at those who err by disagreeing in round 1, we find that these people weigh their own information more than 10 times that of others, putting virtually no stock in public information. This indicates a different type of error, that is, a failure of some individuals to learn from each other. This error is quite large and for a nontrivial minority of the population.

Setting aside the subjects who make systematic errors, we find that individuals still put 50% more weight on their own information than they do on the information revealed through the actions of others, although this difference is not statistically significant. (more)

So in this experiment there is a bottom quintile of idiots, and everyone else seems roughly accurate in discounting the opinions of a pool of others containing such idiots. So in this experiment it seems the main reason people think they are better than others is that everyone, even idiots, don’t think they are idiots. I wonder how behavior would change if everyone was shown clearly that the idiots were no longer participating.

GD Star Rating
a WordPress rating system
Tagged as:
Trackback URL:
  • Kirk

    Interesting to see numbers on something like people’s recognition of inferior thought and discounting of it. Can you post the experiment writeup or paper?

  • vaniver

    Note that odds are good this had college students as subjects- that is, the “quintile” isn’t as descriptive (or optimistic) as we would like.

  • Mark M

    Assuming there is an objectively correct point of view and the same set of information is provided, wouldn’t removing the idiots mean that everyone remaining would agree with each other? In that case, how do you measure the value someone places on their own opinion relative to the value placed on others opinions?

    There was a study a while back, although I don’t have the citation handy, showing that poor performers were also poor at evaluating their own performance, which supports the idea that idiots don’t believe they are idiots.

    • http://daedalus2u.blogspot.com/ daedalus2u

      Mark, if someone is objectively correct about something, then it isn’t an opinion it is an objectively correct fact.

      If you don’t know whether an idea is an objectively correct fact or an unsupported opinion, then either you don’t assign a value judgment to it or you are one of the idiots that needs to be removed.

  • http://www.gwern.net gwern

    “An important paper by Weizsacker (2010) comes to a strikingly similar conclusion, but in a much different context from that studied here. Independently of our study, he shows in a meta-analysis of games with information cascades that a significant share of subjects do not conform to rational expectations, and put about twice the weight on their own information as they do on others. The confluence of his results with our own potentially bolsters both findings.”

  • http://www.gwern.net gwern

    So in this experiment there is a bottom quintile of idiots, and everyone else seems roughly accurate in discounting the opinions of a pool of others containing such idiots. So in this experiment it seems the main reason people think they are better than others is that everyone, even idiots, don’t think they are idiots. I wonder how behavior would change if everyone was shown clearly that the idiots were no longer participating.

    Personally, I’m more interested in what drives the bottom quintile. This was college students apparently, but is it mediated by IQ – they were just too stupid? Or is this more of a type 1/type 2 distinction so you’d find that scores on the CRT instead predict ignoring the public info? Or could we be seeing some sort of misfiring strategy where the bottom quintile always overweight their info (perhaps on some sort of double-or-nothing reasoning – ‘no one will remember when I get it wrong, but if I get it right it will look really impressive!’), so there will always be disagreement with human agents no matter how high their IQ or CRT scores?

  • http://daedalus2u.blogspot.com/ daedalus2u

    The disagreement isn’t predicated on the people being disagreed with being idiots, or even in a minority. Why is there disagreement about global warming? The 95%+ of climate modeling scientists who study it and who are in agreement are not idiots, so why does Santorum disagree with them?

    No doubt a lot of Santorum’s rhetoric is aimed at the idiots of the GOP who are unable to understand the science behind AGW and so is more signaling about tribal loyalty than expressions of knowledge or belief. What Santorum is signaling is that he puts tribal loyalty ahead of being aligned with objective reality.

    But being an AGW denialist hasn’t always been a Conservative or even a GOP position. It has only recently become so, as big money has been poured into AGW denialism by fossil fuel interests. Santorum’s loyalty isn’t to the GOP or Conservatism, it is to the big money that is driving AGW denialism.

    The question is why do some people willingly adopt positions that are completely wrong? Presumably because they consider it in their interest to do so. That was understandable when the Emperor could execute people at a whim, so everyone had to pretend that the Emperor’s new clothes were magnificent.

    The problem is too much power in the hands of idiots like the Emperor.

    • Dave

      It would be un- Dale Carnegie like for me to call anyone an idiot, so I won’t, but here is what you are saying:
      1.) Anyone who does not think that human consumption the earth’s stored carbon fuels that is not offset is causing the earth to permanently warm is an idiot.
      2.) Anyone who does not think that this will lead to disaster is an idiot.
      3.) Anyone who does not believe that immediate drastic action is needed to reduce carbon dioxide and that this will prevent harm that would occur if such action were not taken, is an idiot.
      4.) The windmills, solar panels and renewable fuels programs being built or considered will have an appreciable effect in reducing or slowing global warming. This will appreciably offset increased carbon usage in developing countries such as China and the increased smoke and consumption of wood fuel in undeveloped countries with burgeoning populations and will be a wise way to allocate scarce resourced that could be used otherwise. Only an idiot would disagree.
      5.) This expenditure is best directed by lawmakers who are wise and altruistic and will see to the good of the planet. Anyone who disagrees with this statement is an idiot.
      6.) These actions will not unjustly effect or cause major harm in developed countries or if so this would be preferable to the effects of global warming. Anyone who questions this is an idiot.
      7.) No one knows how much carbon reduction is achievable in what quantity at what expense or is being achieved or is achievable but anyone who does not support drastic action now is an idiot.
      8.) The only things that are slowing actions on these important matters are ignorant, stupid, greedy conservatives and their moneyed cronies who control everything. They are all idiots.

      • http://daedalus2u.blogspot.com/ daedalus2u

        Statements #1, 2, 3, 6, 7, 8 are correct.

        Statement 4 is not correct because it doesn’t go far enough. There are not sufficient AGW mitigation efforts underway. That is the problem.

        Mitigation efforts have to include Developing countries, but it is disingenuous and not useful to tell Developing countries that they may not increase their standard of living, reduce childhood mortality, improve their diets and bring clean water and electricity to their huts while the members of Developed world drive their air-conditioned multi-ton hunks of iron from their air-conditioned McMansions to their air-conditioned Big Box stores instead of walking.

        Statement 5 is not correct because if it were correct then leaders would already be taking action. They are not taking action because of #8. That is the problem.

        Whether there is a problem with AGW or not is not a matter of opinion and is not a question of cost. It is a question of data and physics.

        I don’t think that all AGW deniers are idiots. I reserve the term “idiot” for people who are too stupid to appreciate that they are ignorant and wrong. Ignorance can be fixed, but you can’t fix stupid.

        Many AGW deniers are not idiots, they are simply evil in that they know and understand the problems of AGW, but they are lying about them and denying AGW is a problem because it delivers them profits. They are evil exactly the same way that tobacco executives are evil when they denied the adverse health effects of tobacco so as to get children addicted and raise corporate profits at the expense of children’s lives. Not surprisingly, some of the same players are doing both tobacco denial and AGW denial. They are Merchants of Death. It may be legal but it is still evil.

        Romney and Gingrich are evil in that they know that AGW is real. They once supported doing something about it. Now they are pandering to the useful idiots in the GOP who are too stupid to appreciate that AGW is real. I think Santorum actually believes what he is saying so he is one of the useful idiots. Useful to the evil monied AGW deniers.

      • Dave

        Thanks for the reply. You are clearly expressing your opinion. I will continue reading your posts.

      • http://daedalus2u.blogspot.com/ daedalus2u

        Dave, your statements seemed to be predicated on using a disagreement about AGW to label someone an “idiot”. Labeling someone an idiot (or not) is not my goal or the goal of my statements at all. My goal is for policy makers to have a correct understanding of reality so that a correct understanding of reality will be used to inform policy so that correctly informed policy will lead to a more prosperous and better world.

        My very strong belief is that reality based policy is the best policy, and policies that are reality based have the least total bias against stakeholders. Non-reality based policies favor particular special interests at the expense of others.

        I appreciate that Conservatives view the world differently than I do. I try to view the world through the lens of reality; of facts and logic. Conservatives tend to view the world by believing what people with high social status in that particular social hierarchy tell them. Social status in the Conservative social hierarchy is about position in the hierarchy. In a Conservative social hierarchy, people can be marginalized by high status Conservatives calling them idiots. People can become high status Conservatives by having or paying a lot of money to other Conservatives or by Conservatives saying that they are high status. The nature of Conservatism is to favor the existing social power structure, even (or especially) if it is wrong.

        I don’t disbelieve people because I think they are idiots, I disbelieve people because what they are saying doesn’t correspond with reality and then I infer that they are idiots (or evil) for believing (or pretending) that it does.

        What people believe and what people say has no influence on what actual reality is. Conservatives have a hard time accepting that because their world view is so tied-up with status and signaling on the social status hierarchy, not in trying to figure out what reality is actually like.

  • IVV

    I can imagine that there’s a value to having idiots. Even if they’re wrong and the consensus is right 99.9% of the time, in that final 0.1%, the idiots will be right… if it’s on a survival decision, then someone in the society survives.

    Of course, I’m also reminded of disruptive Wikipedia editors–I remember one who believed that as long as he believed whatever he wanted to believe, there was no consensus, and he could forever enforce his agenda on everything. He was eventually banned.

    I’d be curious about more information about the vehement disagreers and their nature. Are they perceiving information differently? Are they being consciously contrarian? Do they suffer more than average when having to admit being wrong?

    • http://daedalus2u.blogspot.com/ daedalus2u

      IVY, It would only be useful if the idiots were not in charge, and/or were not interfering with the 99+% that are not idiots.

      The problem we have now is that there are idiots in charge and they are interfering with the 99% that are not idiots. The GOP is actively blocking AGW mitigation efforts. Mitigation of AGW will cost much more and be more difficult the longer it is put off.