The blogsphere adores Kruger and Dunning‘s ’99 paper "Unskilled and Unaware of It". Google blog search lists ten blog mentions just in the last month. For example: Perhaps the single academic study most germane to the present election … In short, smart people tend to believe that everyone else "gets it." Incompetent people display both an increasing tendency to overestimate their cognitive abilities and a belief that they are smarter than the majority of those demonstrably sharper.
If the very best you have encountered tend to underestimate their abilities significantly, they are ALSO TERRIBLE at judging where they are in the skills distribution. That's sort of the whole point.
Additionally, it should be noted that Dunning-K does NOT suggest that the less skilled believe they are more competent than the highly-skilled. It's that the less skilled believe they are more competent than they really are. In fact, the less skilled still understood that they were in the lowest quartile of accomplishment -- they just didn't realize how bad they are! So it's not as if you have some loudmouthed idiot who truly believes he is the BEST. It's more a case of the rubbish not understanding just how rubbish they truly are.
Speaking anecdotally, I've generally found that the most highly competent individuals I encountered [in some given domain] were roughly aware of how good they were, but the absolute cream of the crop were neither arrogant or drum-beating about it, nor were they totally wilting flowers. Generally, they tended to be generous with their knowledge and rather confident [in their abilities]. When you can easily bench-press 450lbs, it's pretty hard pretending that you are nowhere near such a strength level. The same applies with skill.
»»» The so-called "refutation" is in fact an extension of the Dunning Kruger effect to extreme cases.«««DK effectively states that lower-skilled people are worse at guessing how good they are at a specific task because they basically don't know any better, being lower-skilled. Whereas the highly-skilled, due to being more competent and knowledgeable, are much better at working out where they are in the skill distribution.
The BLK paper says that this is not the case. So it IS a refutation. It says that everyone is [relatively] bad at guessing how good they are. But due to things like positive self-enhancement impacting the estimation [people tend to rate themselves better than average, on average], the lower-skilled are further away from their place on the curve than the higher-skilled. This happens because the lower-skilled already rate themselves as higher than they are + self-enhancement = far off their accurate place on the curve. Whereas higher-skilled rate themselves lower than they are [still an error] + self-enhancement = closer to their accurate place on the curve.
»»» but I think the main point of the original paper is lost here «««I think you have missed the point of this post, which is that the original paper was flawed in it's conclusions.
»»» If a skilled person miscalculates their competence downward, there is far less harm than if the unskilled overestimates their incompetence upward.«««If you are of low self-esteem, then how do you ever become competent in anything? Everyone starts off as incompetent in the beginning. But those who are adversely affected by negative thoughts, or more sensitive to negative thoughts, will be constantly put off by "realistic" assessments that tell them how awful they are. Here, a little self-delusion, that you have improved, even when that is perhaps not the case or marginal, helps keep you going through the rough patch of training or skills development.
»»»even if the skilled overestimates upward, there is still less harm compared to the unskilled doing the same.««« Possibly not, as we may send the ultra-competent for a sensitive task that requires fine adjustments. If he/she has overestimated how good they are, even though they are highly skilled, we may be put in a dangerous situation. Whereas we will rely on the low-skilled for such sensitive tasks much less. The key here is that we are aware of the high and low skilled individuals. The context counts.
E-money, as far as I can tell, your comment is vague (what precisely does "care" mean?). It may also be massively redundant - it could apply to most of what is done today, even though what we do is still relevant to now. It might even be incorrect; it's not clear to me that these results will stop being relevant in a billion years. Maybe even an advanced typed 3 civilization decides to keep the information around for something. Who knows? How would they? Please clarify.
I am no expert, and I'm sure some here know better than me, but I think the main point of the original paper is lost here. Or at least I look at it a different way. If a skilled person miscalculates their competence downward, there is far less harm than if the unskilled overestimates their incompetence upward. even if the skilled overestimates upward, there is still less harm compared to the unskilled doing the same.
The DK paper accurately describes so many people that I have encountered when they opine on science. They have no current knowledge, but they use proudly display their ignorance.
It seems to me that the DK effect is simply described as intellectual dishonesty.
It seems the D-K effect has claimed another victim. The author of the above.The so-called "refutation" is in fact an extension of the Dunning Kruger effect to extreme cases. It says that there is a bottom of the pit, where even the dumbest sees that he doesn't succeed. Since a simple "I can't" followed by failure is always accurate, while "I may have a chance", followed by occasional failure is less accurate, that's pretty much a commonplace statement.Every theory has it's space of validity. Just because that of the D-K effect doesn't encompass all possible tasks, but just most of the relevant cases, doesn't "refute" the theory.
Thank you for this post. I was recently looking at the 1999 Dunning Kruger study (as part of a research project) and was astounded by its shoddy analysis. It's good to see clearer heads have given the study a re-look.
I am currently considering writing a post on the topic of D-K myself (which is way I stumbled on this post).
Looking at the above, I would caution that there are two big-picture issues to keep in mind, namely:
1. Whether the less competent/intelligent tend to overestimate their abilities: My own observations (and those of many others) point very, very strongly in this direction. Notably, those really sure of themselves have very often turned out to be subaverage in ability, in my experience; while the very best have often been filled with self-doubt.
2. Whether this is explained by low ability in X also leading to a low competence at judging ability in X.
For my part, I was frustrated by 1. long before I encountered the D-K paper and speculated in the reverse causality, most notably that those who were too sure in their abilities failed to develop them further. (Obviously, there is unlikely to be a single explanation.)
Now, if we look at e.g. the blogosphere and ``You're stupid! No, you're stupid!'' discussions, it really does not matter whether 2. is correct---all that is needed is 1. (Which is not to say that the application is typically justified: The ``You disagree; ergo, you are stupid.'' fallacy is quite common---and very hard to avoid when a debate becomes heated.)
With one of Will Roger's aphorisms in mind [ "We're all ignorant, just about different things."], I try very hard to remember that he's talking about me.
I've also learned that one cannot counter emotion with reason. One must start at the emotion and try to move the discussion to reason eventually.
I suspect the reason why nobody has done the study you propose is that variance is a second order effect. ie. You need an order of magnitude more data and/or signal to detect a trend above the noise.
Sad, but that's the hard mathematical facts of statistics.
If the very best you have encountered tend to underestimate their abilities significantly, they are ALSO TERRIBLE at judging where they are in the skills distribution. That's sort of the whole point.
Additionally, it should be noted that Dunning-K does NOT suggest that the less skilled believe they are more competent than the highly-skilled. It's that the less skilled believe they are more competent than they really are. In fact, the less skilled still understood that they were in the lowest quartile of accomplishment -- they just didn't realize how bad they are! So it's not as if you have some loudmouthed idiot who truly believes he is the BEST. It's more a case of the rubbish not understanding just how rubbish they truly are.
Speaking anecdotally, I've generally found that the most highly competent individuals I encountered [in some given domain] were roughly aware of how good they were, but the absolute cream of the crop were neither arrogant or drum-beating about it, nor were they totally wilting flowers. Generally, they tended to be generous with their knowledge and rather confident [in their abilities]. When you can easily bench-press 450lbs, it's pretty hard pretending that you are nowhere near such a strength level. The same applies with skill.
Carl, but that's anecdotal, and we have no idea how reliable a source you are ☺.
»»» The so-called "refutation" is in fact an extension of the Dunning Kruger effect to extreme cases.«««DK effectively states that lower-skilled people are worse at guessing how good they are at a specific task because they basically don't know any better, being lower-skilled. Whereas the highly-skilled, due to being more competent and knowledgeable, are much better at working out where they are in the skill distribution.
The BLK paper says that this is not the case. So it IS a refutation. It says that everyone is [relatively] bad at guessing how good they are. But due to things like positive self-enhancement impacting the estimation [people tend to rate themselves better than average, on average], the lower-skilled are further away from their place on the curve than the higher-skilled. This happens because the lower-skilled already rate themselves as higher than they are + self-enhancement = far off their accurate place on the curve. Whereas higher-skilled rate themselves lower than they are [still an error] + self-enhancement = closer to their accurate place on the curve.
In other words, it is a statistical artefact.
»»» but I think the main point of the original paper is lost here «««I think you have missed the point of this post, which is that the original paper was flawed in it's conclusions.
»»» If a skilled person miscalculates their competence downward, there is far less harm than if the unskilled overestimates their incompetence upward.«««If you are of low self-esteem, then how do you ever become competent in anything? Everyone starts off as incompetent in the beginning. But those who are adversely affected by negative thoughts, or more sensitive to negative thoughts, will be constantly put off by "realistic" assessments that tell them how awful they are. Here, a little self-delusion, that you have improved, even when that is perhaps not the case or marginal, helps keep you going through the rough patch of training or skills development.
»»»even if the skilled overestimates upward, there is still less harm compared to the unskilled doing the same.««« Possibly not, as we may send the ultra-competent for a sensitive task that requires fine adjustments. If he/she has overestimated how good they are, even though they are highly skilled, we may be put in a dangerous situation. Whereas we will rely on the low-skilled for such sensitive tasks much less. The key here is that we are aware of the high and low skilled individuals. The context counts.
You know you asked a guy about something two years ago, right?
(I waited two years to ask you this... please find it funny...)
E-money, as far as I can tell, your comment is vague (what precisely does "care" mean?). It may also be massively redundant - it could apply to most of what is done today, even though what we do is still relevant to now. It might even be incorrect; it's not clear to me that these results will stop being relevant in a billion years. Maybe even an advanced typed 3 civilization decides to keep the information around for something. Who knows? How would they? Please clarify.
In one billion years no one will care about this.
I am no expert, and I'm sure some here know better than me, but I think the main point of the original paper is lost here. Or at least I look at it a different way. If a skilled person miscalculates their competence downward, there is far less harm than if the unskilled overestimates their incompetence upward. even if the skilled overestimates upward, there is still less harm compared to the unskilled doing the same.
The DK paper accurately describes so many people that I have encountered when they opine on science. They have no current knowledge, but they use proudly display their ignorance.
It seems to me that the DK effect is simply described as intellectual dishonesty.
You make the assumption that reason and emotion are mutually exclusive.
You might have D.K. yourself.
It seems the D-K effect has claimed another victim. The author of the above.The so-called "refutation" is in fact an extension of the Dunning Kruger effect to extreme cases. It says that there is a bottom of the pit, where even the dumbest sees that he doesn't succeed. Since a simple "I can't" followed by failure is always accurate, while "I may have a chance", followed by occasional failure is less accurate, that's pretty much a commonplace statement.Every theory has it's space of validity. Just because that of the D-K effect doesn't encompass all possible tasks, but just most of the relevant cases, doesn't "refute" the theory.
Thank you for this post. I was recently looking at the 1999 Dunning Kruger study (as part of a research project) and was astounded by its shoddy analysis. It's good to see clearer heads have given the study a re-look.
I am currently considering writing a post on the topic of D-K myself (which is way I stumbled on this post).
Looking at the above, I would caution that there are two big-picture issues to keep in mind, namely:
1. Whether the less competent/intelligent tend to overestimate their abilities: My own observations (and those of many others) point very, very strongly in this direction. Notably, those really sure of themselves have very often turned out to be subaverage in ability, in my experience; while the very best have often been filled with self-doubt.
2. Whether this is explained by low ability in X also leading to a low competence at judging ability in X.
For my part, I was frustrated by 1. long before I encountered the D-K paper and speculated in the reverse causality, most notably that those who were too sure in their abilities failed to develop them further. (Obviously, there is unlikely to be a single explanation.)
Now, if we look at e.g. the blogosphere and ``You're stupid! No, you're stupid!'' discussions, it really does not matter whether 2. is correct---all that is needed is 1. (Which is not to say that the application is typically justified: The ``You disagree; ergo, you are stupid.'' fallacy is quite common---and very hard to avoid when a debate becomes heated.)
Those who have read Burson, Larrick and Klayman or Krueger and Mueller's response to Kruger and Dunning might be interested in Ehrlinger et al.
Links to all these papers can be found here: http://jdc325.wordpress.com/2009/06/09/incompetence-and-the-general-chiropractic-council/ (scroll down to the last couple of paras to see links to papers).
With one of Will Roger's aphorisms in mind [ "We're all ignorant, just about different things."], I try very hard to remember that he's talking about me.
I've also learned that one cannot counter emotion with reason. One must start at the emotion and try to move the discussion to reason eventually.
I suspect the reason why nobody has done the study you propose is that variance is a second order effect. ie. You need an order of magnitude more data and/or signal to detect a trend above the noise.
Sad, but that's the hard mathematical facts of statistics.