While we tend to be optimistic about our abilities, we are pessimistic about our luck: We analyze the answers of a sample of 1,540 individuals to the following question "Imagine that a coin will be flipped 10 times. Each time, if heads, you win 10C. How many times do you think that you will win?" The average answer is surprisingly about 3.9 which is below the average 5, and we interpret this as a pessimistic bias. We find that women are more "pessimistic" than men, as are old people relative to young.
I wonder if the subjects are being rational, in the sense that they're avoiding the ludic fallacy:http://www.fooledbyrandomne...
Within the rules of the games, yes, they'll win 5, but this exact game they've never played, and more importantly never played with the experimenter, who they cannot 100% trust. So the subjects are just rationally injecting real-world experience or street-smarts into their assessment. I recall an example from when I was 6 years old. I bet a marble with this kid, and I won. Was the game over? Well in a sense, no. The kid cried to teacher, who forced me to give him the marble back. Similarly, if the experimenters don't value the keeping of one's word then they could give the subjects $0 regardless.
P.S. (On OB what's the procedure when you have a new idea about an old post. Is it worth commenting on?)
Jay, Carl, Weldon [the "good luck attributed to skill, therefore people think they have bad luck" faction]: I think I'd buy your explanation if it turned out that the majority of the subjects really try to come up with an estimate based on experience (if somebody came up with a clever way to test for it), but the way I'm imagining the situation, it seems much more likely to me that most subjects confabulate an answer that feels right. Maybe it's because I don't have enough experience with games of luck to form an opinion based on that, and therefore imagine average people don't either? -- On the other hand, I like to play Tali (a Yahtzee-clone) on the computer, trying to beat my own highscores (of all the silly ways to waste time, this must occupy a special place), and I pay attention to the order of types in Tetris, and in both cases I actually get the feeling (not being taken seriously, don't worry) of being lucky above chance; and still I can see myself answering <5 in the study more easily than answering >5.
Dagon's (and Wagster's) position hits closer to my intuitions, for what that's worth...
As a poker player, the results don't surprise me given how the question was phrased. People like to credit good luck to skill and mistakes to bad luck, which is made especially obvious in poker. This leads to a large lucky/unlucky imbalance when people are asked to make an estimate "according to his/her experience and his/her luck".
If you polled professional players whose income relies on knowledge of probabilities and who have seen hundreds of losing players exhibit this bias, and you asked them if their luck has been worse or better than average, I'm sure (and based on my small-sample-sized experience) the vast majority would say they've been unlucky. If people with this much education and awareness regarding this bias are still prone, your average person is hopeless.
As a player, I am grateful on whole for this bias. The fact that luck is often the determining factor in poker short term allows this bias to enable massive self delusion regarding one's skill level. Luck is the perfect scape goat. It can also help people to quit however, as often they get convinced they are permanently unlucky in poker and use this as their reason to stop playing.
I think Dagon is right. My suspicion is that social circumstances are overwhelming rational estimation. If this same question were asked in written form or through a computer terminal, I suspect the pessimism effect would diminish or disappear. Note that the effect was more predominant in women, who are more socially aware.
The concept of "bad luck" requires there to be an ontologically basic mental thing called "luck". Now we all know by now that's not allowed.
What does that leave? Well, there's an old saying: "You are the only common denominator in all your failed relationships." Or you could just be reading too much into chance. Take your pick.
I've recently come to the conclusion that my bad luck is a real phenomenon. I decided that after about the 100th occurrence of this exact template:
Me: I have problem X.People experienced with X: Oh, that's easy! You just have to do Y.Me: *does Y, catastrophically backfires several times over*Me: Um, hey people, I did Y and let me tell you what happened...People experienced with X: No. No. No way. There is no way that could have possibly happened. We defy the data.*after audit of my experience*People: Holy ****! You just got a bad draw there! Well, once in a blue moon something like that happens, just gotta ignore it and move on...Me: !!!
So, am I off-base in drawing the inference I did? It's part of why I seriously considered popping the oil bubble and ending the food riots this summer by going long on oil... Good thing I didn't go through with it or I'd be thoroughly convinced I had magic powers.
Actually, maybe I should call up these researchers, answer their question, and then tell them to go through with the coin tosses where I benefit on heads, and see what the average turns out to be...
I do know someone who seems to have been unusually lucky at coin flips in the Pokemon Trading Card Game, and we joke together about his "Polish Luck". We started doing so when, during one game, he got nine heads in a row. (Incidentally, there's some speculation that the cardboard "coins" that came with the cards are either biased or manipulable.) We don't take it particularly seriously, though, and we've played an awful lot of games, so there's a reasonable chance that someone would hit a lucky streak that had a one-off probability of 1 in 512.
For a lot of these experiments, I strongly suspect that the bias is one of eliciting sympathy. Signalling that you have high capability and low standards due to unfortunate circumstance makes you an ideal trading partner.
And the most reliable signal is one you believe yourself.
So, when it's cheap to underestimate one's luck, one will. I expect you'd see a different result if there were any reason to try to be accurate rather than sympathy-seeking. Say, if you paid based on correctness of estimate rather than on outcome regardless of estimate.
We already know that we tend to attribute our successes and the failures of others to dispositions, and our own failures and others' successes to circumstances/luck. Since most people are not unusually successful, they will wind up attributing an unusual amount of bad luck to themselves.
- The study consisted of face-to-face interviews in the field by "professional interviewers experienced for in-person surveys" (p.4);
- The question was hypothetical;
- "The participant is then asked for his/her own estimation, according to his/her experience and his/her luck, of the number of times heads will occur" (p.4, emph mine);
- On a sample of thirty, if there is no gain associated to the coin tossing, the average is 4.9, and 90% answer 5 (p.3).
It does sound to me as if the way the question was framed would have given the impression that the interviewer wanted to hear an answer based on an (in Carrier's terms) supernatural concept of personal luck. Of course I don't expect that Eliezer would have been swayed :-) but it seems likely to me that there was a degree of please-the-researcher, leading people to give estimates conditional on supernatural personal luck existing.
Of course, it's also plausible that without such a prime, people might infer that the researcher wants to hear the rational result. But even if that justifies the method, it doesn't make the present results stronger... (I wonder whether the question was framed the same way in the sample without winnings. Anyone know if the conventions of the field make this implied by default?)
The direction of the deviation is interesting to me -- I'm not sure I could have predicted it -- but it still seems more likely to me to be a result of the experimental situation than a reliable revelation of expectations in actual games. If people haven't considered the question before, it seems like an invitation to confabulate the result that will make the subject look best. It seems like a high estimate would look better if the interviewer believed it, but worse if it did not sound credible, so it seems plausible that people would give the highest answer that they can feel relatively sure of attaining.
It's a bit hard to square this interpretation with the general trend towards overconfidence, though. Maybe because it is very easy to do an objective test, and very hard to influence the results, so personal luck, if it did exist, would be a particularly hard-to-fake ability? (But perhaps that's a bit too detailed an explanation to be particularly likely?)
Despair is so tempting! One interesting feature of the modern intarweb is that most mainstream news sites now allow ordinary citizens to comment on the stories of the day. Sometimes I go to these sites (like cnn, msnbc, or fox) and read the comment threads on political or economic stories.
People are certainly much different than the "striving to be objective, rational, kind, and honest" creatures I want them to be. I'm not even sure whether I can really be that different... perhaps I just fool myself.
If Eliezer is unable to convince Robin and most onlookers here (among the most rational people you'll find, all answering 5 to the topic question) of his probability vs time distribution estimate for AI hard takeoff, what chance does he have with the bulk of humanity? Surely rhetorical tricks, "ai-baiting", or religious fervor will be more likely to succeed. Would the ends justify such means?
Maybe even trying is fruitless and a recruitment + organization + implementation effort for building Friendly AI should just proceed without worrying about what most people think.
I wonder if the subjects are being rational, in the sense that they're avoiding the ludic fallacy:http://www.fooledbyrandomne...
Within the rules of the games, yes, they'll win 5, but this exact game they've never played, and more importantly never played with the experimenter, who they cannot 100% trust. So the subjects are just rationally injecting real-world experience or street-smarts into their assessment. I recall an example from when I was 6 years old. I bet a marble with this kid, and I won. Was the game over? Well in a sense, no. The kid cried to teacher, who forced me to give him the marble back. Similarly, if the experimenters don't value the keeping of one's word then they could give the subjects $0 regardless.
P.S. (On OB what's the procedure when you have a new idea about an old post. Is it worth commenting on?)
Jay, Carl, Weldon [the "good luck attributed to skill, therefore people think they have bad luck" faction]: I think I'd buy your explanation if it turned out that the majority of the subjects really try to come up with an estimate based on experience (if somebody came up with a clever way to test for it), but the way I'm imagining the situation, it seems much more likely to me that most subjects confabulate an answer that feels right. Maybe it's because I don't have enough experience with games of luck to form an opinion based on that, and therefore imagine average people don't either? -- On the other hand, I like to play Tali (a Yahtzee-clone) on the computer, trying to beat my own highscores (of all the silly ways to waste time, this must occupy a special place), and I pay attention to the order of types in Tetris, and in both cases I actually get the feeling (not being taken seriously, don't worry) of being lucky above chance; and still I can see myself answering <5 in the study more easily than answering >5.
Dagon's (and Wagster's) position hits closer to my intuitions, for what that's worth...
Isn't this related to risk-aversion somehow?
As a poker player, the results don't surprise me given how the question was phrased. People like to credit good luck to skill and mistakes to bad luck, which is made especially obvious in poker. This leads to a large lucky/unlucky imbalance when people are asked to make an estimate "according to his/her experience and his/her luck".
If you polled professional players whose income relies on knowledge of probabilities and who have seen hundreds of losing players exhibit this bias, and you asked them if their luck has been worse or better than average, I'm sure (and based on my small-sample-sized experience) the vast majority would say they've been unlucky. If people with this much education and awareness regarding this bias are still prone, your average person is hopeless.
As a player, I am grateful on whole for this bias. The fact that luck is often the determining factor in poker short term allows this bias to enable massive self delusion regarding one's skill level. Luck is the perfect scape goat. It can also help people to quit however, as often they get convinced they are permanently unlucky in poker and use this as their reason to stop playing.
long time to develop. Sorry about the broken comment.
Like the epsilon-delta definition of a limit, the theory of expected value took a long time to develop.
I think Dagon is right. My suspicion is that social circumstances are overwhelming rational estimation. If this same question were asked in written form or through a computer terminal, I suspect the pessimism effect would diminish or disappear. Note that the effect was more predominant in women, who are more socially aware.
Silas: Perhaps you tend to rely on wrong authorities on various subjects?
You. Do. Not. Have. Bad. Luck.
The concept of "bad luck" requires there to be an ontologically basic mental thing called "luck". Now we all know by now that's not allowed.
What does that leave? Well, there's an old saying: "You are the only common denominator in all your failed relationships." Or you could just be reading too much into chance. Take your pick.
I've recently come to the conclusion that my bad luck is a real phenomenon. I decided that after about the 100th occurrence of this exact template:
Me: I have problem X.People experienced with X: Oh, that's easy! You just have to do Y.Me: *does Y, catastrophically backfires several times over*Me: Um, hey people, I did Y and let me tell you what happened...People experienced with X: No. No. No way. There is no way that could have possibly happened. We defy the data.*after audit of my experience*People: Holy ****! You just got a bad draw there! Well, once in a blue moon something like that happens, just gotta ignore it and move on...Me: !!!
So, am I off-base in drawing the inference I did? It's part of why I seriously considered popping the oil bubble and ending the food riots this summer by going long on oil... Good thing I didn't go through with it or I'd be thoroughly convinced I had magic powers.
Actually, maybe I should call up these researchers, answer their question, and then tell them to go through with the coin tosses where I benefit on heads, and see what the average turns out to be...
I do know someone who seems to have been unusually lucky at coin flips in the Pokemon Trading Card Game, and we joke together about his "Polish Luck". We started doing so when, during one game, he got nine heads in a row. (Incidentally, there's some speculation that the cardboard "coins" that came with the cards are either biased or manipulable.) We don't take it particularly seriously, though, and we've played an awful lot of games, so there's a reasonable chance that someone would hit a lucky streak that had a one-off probability of 1 in 512.
For a lot of these experiments, I strongly suspect that the bias is one of eliciting sympathy. Signalling that you have high capability and low standards due to unfortunate circumstance makes you an ideal trading partner.
And the most reliable signal is one you believe yourself.
So, when it's cheap to underestimate one's luck, one will. I expect you'd see a different result if there were any reason to try to be accurate rather than sympathy-seeking. Say, if you paid based on correctness of estimate rather than on outcome regardless of estimate.
We already know that we tend to attribute our successes and the failures of others to dispositions, and our own failures and others' successes to circumstances/luck. Since most people are not unusually successful, they will wind up attributing an unusual amount of bad luck to themselves.
if there is no gain associated to the coin tossing, the average is 4.9
This is the scariest bit. If you take ten pence from them when they lose, does the result fly up over 5?
Benja, you're no doubt right about the nature of the question, but let's be honest here - anything but 5 is unforgivable.
Note that
- Robin's "10C" means 10 euros;
- The study consisted of face-to-face interviews in the field by "professional interviewers experienced for in-person surveys" (p.4);
- The question was hypothetical;
- "The participant is then asked for his/her own estimation, according to his/her experience and his/her luck, of the number of times heads will occur" (p.4, emph mine);
- On a sample of thirty, if there is no gain associated to the coin tossing, the average is 4.9, and 90% answer 5 (p.3).
It does sound to me as if the way the question was framed would have given the impression that the interviewer wanted to hear an answer based on an (in Carrier's terms) supernatural concept of personal luck. Of course I don't expect that Eliezer would have been swayed :-) but it seems likely to me that there was a degree of please-the-researcher, leading people to give estimates conditional on supernatural personal luck existing.
Of course, it's also plausible that without such a prime, people might infer that the researcher wants to hear the rational result. But even if that justifies the method, it doesn't make the present results stronger... (I wonder whether the question was framed the same way in the sample without winnings. Anyone know if the conventions of the field make this implied by default?)
The direction of the deviation is interesting to me -- I'm not sure I could have predicted it -- but it still seems more likely to me to be a result of the experimental situation than a reliable revelation of expectations in actual games. If people haven't considered the question before, it seems like an invitation to confabulate the result that will make the subject look best. It seems like a high estimate would look better if the interviewer believed it, but worse if it did not sound credible, so it seems plausible that people would give the highest answer that they can feel relatively sure of attaining.
It's a bit hard to square this interpretation with the general trend towards overconfidence, though. Maybe because it is very easy to do an objective test, and very hard to influence the results, so personal luck, if it did exist, would be a particularly hard-to-fake ability? (But perhaps that's a bit too detailed an explanation to be particularly likely?)
Despair is so tempting! One interesting feature of the modern intarweb is that most mainstream news sites now allow ordinary citizens to comment on the stories of the day. Sometimes I go to these sites (like cnn, msnbc, or fox) and read the comment threads on political or economic stories.
People are certainly much different than the "striving to be objective, rational, kind, and honest" creatures I want them to be. I'm not even sure whether I can really be that different... perhaps I just fool myself.
If Eliezer is unable to convince Robin and most onlookers here (among the most rational people you'll find, all answering 5 to the topic question) of his probability vs time distribution estimate for AI hard takeoff, what chance does he have with the bulk of humanity? Surely rhetorical tricks, "ai-baiting", or religious fervor will be more likely to succeed. Would the ends justify such means?
Maybe even trying is fruitless and a recruitment + organization + implementation effort for building Friendly AI should just proceed without worrying about what most people think.