In Moore’s experiment, volunteers were given cash for correctly guessing the weight of people from their photographs. In each of the eight rounds of the study, the guessers bought advice from one of four other volunteers. The guessers could see in advance how confident each of these advisers was (see table), but not which weights they had opted for.
From the start, the more confident advisers found more buyers for their advice, and this caused the advisers to give answers that were more and more precise as the game progressed. This escalation in precision disappeared when guessers simply had to choose whether or not to buy the advice of a single adviser. In the later rounds, guessers tended to avoid advisers who had been wrong previously, but this effect was more than outweighed by the bias towards confidence. …
Moore said that following the advice of the most confident person often makes sense, as there is evidence that precision and expertise do tend to go hand in hand. For example, people give a narrower range of answers when asked about subjects with which they are more familiar.
There are times, however, when this link breaks down. With complex but politicised subjects such as global warming, for example, scientific experts who stress uncertainties lose out to activists or lobbyists with a more emphatic message.
That is from New Scientist back in June. So the key hard question is: when does confidence credibly signal expertise, and when it is just empty cheap talk? Hat tip to Toby Ord.
Gladwell:
The psychologist Ellen Langer once had subjects engage in a betting game against either a self-assured, well-dressed opponent or a shy and badly dressed opponent (in Langer’s delightful phrasing, the “dapper” or the “schnook” condition), and she found that her subjects bet far more aggressively when they played against the schnook. They looked at their awkward opponent and thought, I’m better than he is. Yet the game was pure chance: all the players did was draw cards at random from a deck, and see who had the high hand. This is called the “illusion of control”: confidence spills over from areas where it may be warranted (“I’m savvier than that schnook”) to areas where it isn’t warranted at all (“and that means I’m going to draw higher cards”).
I was once a shy badly dressed person who didn’t understand why people kept underestimating him. Now I know.
The US public has long felt guilty about historical racism, and so they felt great about electing a black president, and giving him some space to do what he wants. That and Bush-hatred seemed to give progressives a rare opportunity to achieve many things they’ve always wanted.
This opportunity now seems substantially squandered. Obama has spent much of his political capital on changes far from professed progressive priorities. He spent lots on fiscal stimulus and nationalizing the auto industry, and seems to have too little left for big medical and financial reform. Tyler and I have long said Obama can’t do near as much as supporters seemed to think, and we’ve been puzzled to hear this prospect dismissed out of hand .
For Obama and his new-to-power advisers, this failure may come from overconfidence, but his old-hand-operative advisers should have known better. So either their stern warnings were ignored, or they betrayed him by not making stern warnings. Since much of Obama’s political capital was spent by longtime congressional Democrats choosing the detailed pork of stimulus and related bills, they might be blamed. If they said “Support us to pick this pork now, and we’ll help you get health care reform later,” they were basically taking him to the cleaners.
For progressives, the lost opportunity here is not just bills not passed, or the public checking off “elect a minority president” from their to-do-list. Another cost is a more conservative public opinion. Psychology experiments explain: Continue Reading "Obama’s Opportunity Cost" »
Let's say you have been promoting some view (on some complex or fraught topic – e.g. politics, religion; or any "cause" or "-ism") for some time. When somebody criticizes this view, you spring to its defense. You find that you can easily refute most objections, and this increases your confidence. The view might originally have represented your best understanding of the topic. Subsequently you have gained more evidence, experience, and insight; yet the original view is never seriously reconsidered. You tell yourself that you remain objective and open-minded, but in fact your brain has stopped looking and listening for alternatives.
Here is a debiasing technique one might try: writing a hypothetical apostasy. Remind yourself before you start that unless you later choose to do so, you will never have to show this text to anyone.
Imagine, if you will, that the world's destruction is at stake and the only way to save it is for you to write a one-pager that convinces a jury that your old cherished view is mistaken or at least seriously incomplete. The more inadequate the jury thinks your old cherished view is, the greater the chances that the world is saved. The catch is that the jury consists of earlier stages of yourself (such as yourself such as you were one year ago). Moreover, the jury believes that you have been bribed to write your apostasy; so any assurances of the form "trust me, I am older and know better" will be ineffective. Your only hope of saving the world is by writing an apostasy that will make the jury recognize how flawed/partial/shallow/juvenile/crude/irresponsible/incomplete and generally inadequate your old cherished view is.
(If anybody tries this, feel free to comment below on whether you found the exersise fruitful or not – but no need to state which specific view you were considering or how it changed.)
New data question the claim that people tend to overestimate their abilities:
A large body of literature purports to find that people are generally overconfident. In particular, a better-than-average effect in which a majority of people claim to be superior to the average person has been noted for a wide range of skills, from driving, to spoken expression, to the ability to get along with others, to test taking on simple tests. The literature generally accepts that this better-than-average effect is indicative of inflated self- assessments. However, [we] recently … show that the better-than-average data … does not indicate … people have made some kind of error in their self-evaluations. Because of this reason, almost none of the existing experimental literature on relative overconfidence can actually claim to have found overconfidence. … In this paper, we report on an experiment designed to provide a proper test of overconfidence. … As in much previous experimental work, we find a better-than-average effect among our subjects. … We find evidence that subjects are uncertain of their own types. Our experiment can be viewed as a test of the null hypothesis that people are behaving rationally (and are not overconfident). We cannot reject that hypothesis.
Not too long ago, the well-known economist Robert Hall presented this paper (co-authored with Susan E. Woodward) at my place of work. Here is the abstract:
In the standard venture capital contract, entrepreneurs have a large fraction of equity ownership in the companies they found and are paid a sub-market salary by the investors who provide the money to develop the idea. The big rewards come only to those whose companies go public or are acquired on favorable terms, forcing entrepreneurs to bear a substantial burden of idiosyncratic risk. We study this burden in the case of high-tech companies funded by venture capital. Over the past 20 years, the typical venture-backed entrepreneur earned an average of $4.4 million from companies that succeeded in attracting venture funding. Entrepreneurs with a coefficient of relative risk aversion of two and with less than $0.7 million would be better off in a salaried position than in a startup, despite the prospect of an average personal payoff of $4.4 million and the possibility of payoffs over $1 billion. We conclude that startups attract entrepreneurs with lower risk aversion, higher initial assets, preferences for entrepreneurship over employment, and optimistic beliefs about the payoffs from their products.
During the seminar it occurred to me that these results, assuming they are correct, are evidence of an absence of overconfidence, at least among the kinds of people who leave good jobs to form high-tech startups. The reason is that if potential entrepreneurs were massively overconfident, one would expect to see lots of entry of startups based on weak ideas, which would lead to an expected payoff so low that forming a startup would be a losing proposition for the potential entrepreneur unless he/she started out extremely wealthy and/or had very low risk-aversion. But what the authors actually find is that forming a startup with an average-quality idea* is a break-even proposition for a potential entrepreneur with quite modest wealth and with a more-or-less standard degree of risk-aversion.
After the talk, I asked Professor Hall if he agreed with this interpretation (he seemed to), and if he would object to my posting about it on OB (he didn't). But I will make him aware of this post, and invite him to comment if he would like, and correct any mistakes that I might have made.
*The authors have no way to distinguish the quality of an idea, so there is an implicit assumption that the marginal quality of the idea is equal to the average quality of all ideas that actually get implemented.
While we tend to be optimistic about our abilities, we are pessimistic about our luck:
We analyze the answers of a sample of 1,540 individuals to the following question "Imagine that a coin will be flipped 10 times. Each time, if heads, you win 10C. How many times do you think that you will win?" The average answer is surprisingly about 3.9 which is below the average 5, and we interpret this as a pessimistic bias. We find that women are more "pessimistic" than men, as are old people relative to young.
Added: Benja Fallenstein notes "if there is no [personal] gain associated to the coin tossing, the average [guess] is 4.9, and 90% answer 5.
We are more overconfident on tasks we don’t actually expect to perform, and when we don’t expect to have to explain our evaluation to others. On expecting to perform:
Participants made predictions about performance on tasks that they did or did not expect to complete. In three experiments, participants in task-unexpected conditions were unrealistically optimistic: They overestimated how well they would perform, often by a large margin, and their predictions were not correlated with their performance. By contrast, participants assigned to task-expected conditions made predictions that were not only less optimistic but strikingly accurate. Consistent with predictions from construal level theory, data from a fourth experiment suggest that it is the uncertainty associated with hypothetical tasks, and not a lack of cognitive processing, that frees people to make optimistic prediction errors. Unrealistic optimism, when it occurs, may be truly unrealistic; however, it may be less ubiquitous than has been previously suggested.
On expecting to explain:
Accountability … [is] the expectation to explain, justify, and defend one’s self-evaluations (grades on an essay) to another person ("audience"). Experiment 1 showed that accountability curtails self-enhancement. Experiment 2 ruled out audience concreteness and status as explanations for this effect. Experiment 3 demonstrated that accountability-induced self-enhancement reduction is due to identifiability. Experiment 4 documented that identifiability decreases self-enhancement because of evaluation expectancy and an accompanying focus on one’s weaknesses.
It is almost as if we at some level realize that our overconfidence is unrealistic.
The latest Journal of Prediction Markets has a lit review on overconfidence, with a to-me surprising result: financial overconfidence increases with age, experience, and success. Here are three investment experiments:
Kirchler and Maciejovsky (2002) …. demonstrated that subjects were overconfident in late trading periods … [but] underconfident during other trading periods. … Dittrich et al (2005) … found that age was positively correlated to overconfidence for complex tasks. … Glaser et al (2005) … [found that financial market] professionals’ degrees of overconfidence was higher than that of the student subjects in most tasks, and it appeared that was because the "professionals are biased in job related tasks, such as forecasting real world financial time series."
Also:
A survey of German stock market forecasters conducted by Deaves et al (2005) … demonstrated that the market forecasters were overconfident in their predictions and that greater market experience and success, measured by correct predictions, increased their overconfidence. … Markets are likely to become more overconfident when market returns are high.
This is disturbing. If overconfidence is positively correlated with ability, then observers can rationally take overconfidence as a signal of ability, and people can want to appear more overconfident to appear more able. But to make this story work, somehow it should on average be easier to get away with being more overconfident when one is more experienced and successful. How can this be?
This all seems to make it more reasonable than one might otherwise have thought to disagree about finance with older, more experienced, more successful folks.
More on our overconfident kids from a thoughful essay by Joseph Epstein:
So often in my literature classes students told me what they "felt" about a novel, or a particular character in a novel. I tried, ever so gently, to tell them that no one cared what they felt; the trick was to discover not one’s feelings but what the author had put into the book, its moral weight and its resultant power. In essay courses, many of these same students turned in papers upon which I wished to–but did not–write: "D-, Too much love in the home." I knew where they came by their sense of their own deep significance and that this sense was utterly false to any conceivable reality. Despite what their parents had been telling them from the very outset of their lives, they were not significant. Significance has to be earned, and it is earned only through achievement. Besides, one of the first things that people who really are significant seem to know is that, in the grander scheme, they are themselves really quite insignificant.
"
