The blogsphere adores Kruger and Dunning‘s ’99 paper "Unskilled and Unaware of It". Google blog search lists ten blog mentions just in the last month. For example:
Perhaps the single academic study most germane to the present election … In short, smart people tend to believe that everyone else "gets it." Incompetent people display both an increasing tendency to overestimate their cognitive abilities and a belief that they are smarter than the majority of those demonstrably sharper.
This paper describes everyone’s favorite theory of those they disagree with, that they are hopelessly confused idiots unable to see they are idiots; no point in listening to or reasoning with such fools. However, many psychologists have noted Kruger and Dunning’s main data is better explained by positing simply that we all have noisy estimates of our ability and of task difficulty. For example, Burson, Larrick, and Klayman’s ’06 paper "Skilled or Unskilled, but Still Unaware of It":
We replicated, eliminated, or reversed the association between task performance and judgment accuracy reported by Kruger and Dunning (1999) depending on task difficulty. On easy tasks, where there is a positive bias, the best performers are also the most accurate in estimating their standing, but on difficult tasks, where there is a negative bias, the worst performers are the most accurate. This pattern is consistent with a combination of noisy estimates and overall bias, with no need to invoke differences in metacognitive abilities. In this regard, our findings support Krueger and Mueller’s (2002) reinterpretation of Kruger and Dunning’s (1999) findings. An association between task-related skills and metacognitive insight may indeed exist, and later we offer some additional tests using the current data. However, our analyses indicate that the primary drivers of miscalibration in judging percentile are general inaccuracy due to noise and overall biases that arise from task difficulty.
So why does Google blog search finds zero mentions of this refutation? My guess: because under this theory you should listen to those you disagree with instead of writing them off as idiots.
Now Kruger and Dunning do have a 2008 followup paper, and in their first paper they were able to construct one situation where more able people had lower errors in estimating their ability. Also, Burson et al. saw some weak tendencies like this:
We regressed perceived percentile on actual percentile among bottom-half performers and among top half performers. A Chow test confirmed at a marginal level of significance that bottom-half performers were less sensitive to their actual percentile … than were top-half performers.
Oddly, none of the dozen papers on this I’ve read pursue the obvious way to settle this question: look at the variance of ability estimates as a function of ability. But however that turns out it seems clear that mostly what is going on is that we all misjudge our ability and task difficulty.
loading...
Hal 
Cameron Taylor 
frelkins 
Doug S. 
WWWebb 
John Carter 
Ralph S. Sturgen 
Carl Malone 
Julia 
Kai Frederking 
E-money 
[...] Fall, weil ich glauben möchte, dass das Festlegen und Aussprechen von Verboten gut gemeint ist und bloß das Wissen fehlt, anders und im besten Fall richtig zu handeln. Zwar ist bekanntlich der Weg zur Hölle gepflastert [...]
By Weil nicht sein kann, was nicht sein darf › Immersion I/O August 19, 2009 at 4:28 pm
[...] a perceived difference between the knowledgeable and, well, everyone else. But as it turns out, there’s some debate over whether the effect is actually what it seems, so I’m forced to adjust my rant [...]
By Woodworking and the Dunning-Kruger Effect | Tommy Mac Team Blog November 19, 2010 at 10:48 am
[...] Back in ‘08 I wrote: [...]
By Overcoming Bias : The Unskilled Are Aware March 4, 2011 at 2:24 pm
[...] Robin Hanson has had another stab at the oft-quoted paper by Kruger and Dunning, Unskilled and Unaware of It. The first couple of sentences of the paper’s abstract gives Kruger and Dunning’s basic (and somewhat amusing) claim: People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. [...]
By Unskilled and unaware — Evolving Economics March 7, 2011 at 8:45 am
[...] All are skill unaware. [...]
By Paul van Buuren » Blog archief » Dunning-Kruger-effect May 24, 2011 at 3:11 am
[...] How Dunning-Kruger is often misunderstood [...]
By The Dunning-Kruger Effect « You Are Not So Smart September 14, 2011 at 6:58 pm
[...] : This post was heavily influenced by “All Are Skill Unaware” by Robin Hanson. Related posts [...]
By Equally Unaware Of Stupidity | W-Shadow.com November 13, 2011 at 4:24 am