From the current New Scientist: In an experiment dubbed "Cola Wars", [Nick Epley] conducted a taste test with a twist: he told participants which cola was Coke and which was Pepsi before tasting began. After tasting, all they had to do was estimate what percentage of their friends would be able to distinguish between the two in a blind taste test. Studies show that people’s ability to do this is no better than chance – so an answer around 50 per cent would be right. What Epley found was intriguing. When he motivated volunteers to give a considered response – by offering them a cash payment – their answers tended to be close to 50 per cent. Subjects who were not paid, however, seemed to answer with an egocentric bias: since they knew which cola was which, they assumed that a high proportion of their friends would guess correctly (
I'm reading "Blink" by Malcolm Gladwell, and he discusses something that seems relevant here. The sip test with the challenge of distinguishing two colas isn't a very effective test of flavour discrimination. The test that really distinguishes flavor experts from non-experts is the "triangle test": three glasses, two of which contain the same cola. The challenge is drink them in sequence (presumably with a palate cleanser in between) and then pick the odd one out. Research shows that the general population can't do much better than chance at this test -- you really need to be an expert to perform well.
Cognition,There was a problem from a math club that a guy brought into work and all the math people where working on it for days. He and one other person got it and he got it first. There where some smart people working there. Also one day I was working on a math problem, probably not difficult for a math person but for a regular programmer like me it was not something that I knew and I was having trouble finding how to do it. I asked him and off the top of his head he wrote to ways to do it on white board. It also was not in his area having to do with finance rather than science which was more in his area. I was impressed but maybe just because I am a dolt.
I'm pretty confident in my assessment of my own abilities.
It would not surprise me if there are huge numbers of people who believe they can tell the difference but, in fact, cannot. But I'm quite sure that there is a minority (perhaps small) of people who can pretty reliably tell. I've done the blind test before successfully.
N=27 explains a lot. If, say, 10% of the population has the ability to distinguish semi-reliably, and 90% do not, then it would be easy to have the data of 2-3 partcipants who can distinguish be overwhelmed by some variance in the data from the 24-25 who can't. Also the chance that a sample of 27 contains nobody who can distinguish would be small but significant -- around 6%.
Although the guy we tested seemed to be able to tell the difference between Coke and Pepsi I would still be surprised to learn that many others could do the same. The guy we tested had strange habit and abilities. He drank a lot of coke and was able to list like 50 digits of pi and do some awesome math off the top of his head and yet was unable to hold a job.
Psychohistorian, subjects were told the payment (a $50 gift certificate) would go to the person who came closest to guessing the right answer, so that should motivate subjects to get it right rather than to say what the experimenter wanted to hear. (The paper doesn't tell whether the winner was actually paid!)
Based on the skeptical comments here, I looked for articles on how accurate people actually are at guessing Coke vs Pepsi. Apparently it is a widely assigned student project but I couldn't find much in the way of rigorous studies on it. The paper linked in the article says they did test people to see how they would do:
"we assumed that informing participants of the drinks' identities ahead of time would lead them to taste a difference between them - a difference that would not be apparent to those uninformed of the identities. Indeed, in a pilot test of this assumption, 85% of students in the condition informed of the drinks' identities (N = 27) believed they could have correctly distinguished between the drinks even if they had not been informed - a figure that differed substantially from the 50% accuracy rate actually obtained from a sample of uninformed participants"
So only a 50% accuracy rate among people who weren't told which was which.
Another paper I found was mostly about testing people in an MRI machine to see how their brains lit up while tasting the drinks, but also did some studies of drink preferences outside the machine as controls. These people were not asked to guess Coke vs Pepsi per se, but they indicated which drink they habitually preferred, and then were asked to choose the drink they liked best from tasting two unlabeled cups. They made them do the taste comparison 15 times. Figure 1D shows the results: among people who preferred Coke, they picked the Coke cup 7.5 times out of 15 on average; Pepsi drinkers picked the Coke 6.8 times out of 15. So there was a small difference in the expected direction, but it was not statistically significant. The bottom line is that close to 50% of people picked the wrong drink.
The confidence among posters here is therefore rather surprising. Either it represents overconfidence bias, or else the article has prompted a selection effect so that people who are unusually adept at perceiving these taste differences are motivated to comment.
This may be more of a "guessing the password" answer. When people are asked without the cash incentive, they offer an honest (though possibly biased) estimate. When they have the cash incentive, they guess the answer they think you want to hear. It coincides with the truth in this case, but that is largely coincidental.
I'd bet that if you predisposed them to some (arbitrary) hypothesis, say from the researchers, say, 37%, the cash incentive would cause their responses to cluster more tightly around that point than those without the cash incentive,and it would have nothing to do with the truth.
I once conducted a double blind taste test using coke and pepsi with my sister as the taster and someone else to do the blinding. She said she loved coke and hated pepsi, I predicted she would be unable to distinguish them, she was able to but she repeatedly thought the pepsi was the coke and vise versa. After the findings were reported she actually said something like "Well I like coke cause it's american." - I feel like that was the best thing I've ever done in terms of a clear example of the usefulness of modern scientific testing to regular people.
My business partner and I thought that a person could not tell the difference between Coke and Pepsi but a guy working in the next office bet us that he could. Money was wagered. To make it more difficult we included some store brands and RC cola, which he insisted he could easy tell where not coke or Pepsi. He had most difficulty distinguishing between RC and Pepsi but in the end he identified Coke and Pepsi correctly and the store brands and RC as not coke or Pepsi and we paid. Now I must say that I could never do that.
Way of topic but this is not the first bit of independent research that I have done, I was working managing a restaurant at the time of the big debate about government provided daycare and we had a good number of single moms working in the restaurant so asked them would they rather have free daycare for their children or $10, $20, $30, $40, $50 given to them per week. It turned out that most had arrangements with family or friends and would take less than $20/week above free day care. Some said that they would not use the day care at all even if it was provided free to them.
Hugo, I can't imagine any treatment would make rationality violations disappear. Getting better performance on judgment tasks is exactly what I'm talking about.
Don't dismiss the 50-50 result so quickly. I also find Coke and Pepsi to taste different. But most people, myself included, cannot distinguish between a *sip* of each, particularly when the beverages are cold. It's an easy test to do on a reasonable sample.
My issue with the cited study is more likely to be whether the results actually stem from an egocentric bias. I have not read the study but too much research shoehorns the findings into the authors' pet theories. I want to believe the higher level conclusion that people make better decisions when the decisions matter but the cited study does not seem like the first experiment that I'd design to test it.
but see:Camerer et Hogarth 1999 The Effects of Financial Incentives in Experiments A Review and Capital-Labor-Production Frameworkabstract:We review 74 experiments with no, low, or high performance-based financial incentives. The modal result is no effect on mean performance though variance is usually reduced by higher payment.. Higher incentive does improve performance often, typically judgment tasks that are responsive to better effort. Incentives also reduce ‘‘presentation’’ effects e.g., generosity and risk-seeking.. Incentive effects are comparable to effects of other variables, particularly ‘‘cognitive capital’’ and task ‘‘production’’ demands,and interact with those variables, so a narrow-minded focus on incentives alone is misguided. We also note that no replicated study has made rationality violations disappear purely by raising incentives.
Let's repeat that last sentence:We also note that no replicated study has made rationality violations disappear purely by raising incentives.
Note, I am defying the data of these studies "Studies show that people's ability to do this is no better than chance", not the new one.
I actually think the ability of the whole population to do this is probably very little better than chance, but only because most people have never drunk either often enough to know the taste.
This notion that you can't tell the difference is ridiculous, unless your subject population rarely drinks either. I've been served coke having ordered a pepsi, and I know it. It just doesn't taste the same.
I'd be willing to bet a lot of money on my ability to tell coke from pepsi in blind taste tests with a disinterested labeler, and I rarely drink cola anymore.
celeriac's results don't surprise me in the least. I would expect anyone who drank one or the other regularly and expressed a strong preference to be able to tell the difference pretty reliably (at least 80% chance). I expect I'd get pretty close to 100% under clean trial conditions (bottled not fountain, served in clean glass glasses).
Know of any research that illuminates subjectively experienced thought processes involved in ego-centric mode and analytical mode? Protocol analysis of some sort for instance?
"Take this study designed to see how we judge trustworthiness. Lisa DeBruine at the University of Aberdeen in the UK had volunteers play an online game in which they had to decide whether to trust another player to split a sum of money. All they had to go on was a picture. Volunteers were shown photos of students at another university, and also saw photos that DeBruine had manipulated electronically to incorporate aspects of the volunteers' own faces. The manipulations were subtle enough for the volunteers not to notice them but the effect was remarkable nevertheless, with players far more likely to trust the face."
I'm reading "Blink" by Malcolm Gladwell, and he discusses something that seems relevant here. The sip test with the challenge of distinguishing two colas isn't a very effective test of flavour discrimination. The test that really distinguishes flavor experts from non-experts is the "triangle test": three glasses, two of which contain the same cola. The challenge is drink them in sequence (presumably with a palate cleanser in between) and then pick the odd one out. Research shows that the general population can't do much better than chance at this test -- you really need to be an expert to perform well.
Cognition,There was a problem from a math club that a guy brought into work and all the math people where working on it for days. He and one other person got it and he got it first. There where some smart people working there. Also one day I was working on a math problem, probably not difficult for a math person but for a regular programmer like me it was not something that I knew and I was having trouble finding how to do it. I asked him and off the top of his head he wrote to ways to do it on white board. It also was not in his area having to do with finance rather than science which was more in his area. I was impressed but maybe just because I am a dolt.
I'm pretty confident in my assessment of my own abilities.
It would not surprise me if there are huge numbers of people who believe they can tell the difference but, in fact, cannot. But I'm quite sure that there is a minority (perhaps small) of people who can pretty reliably tell. I've done the blind test before successfully.
N=27 explains a lot. If, say, 10% of the population has the ability to distinguish semi-reliably, and 90% do not, then it would be easy to have the data of 2-3 partcipants who can distinguish be overwhelmed by some variance in the data from the 24-25 who can't. Also the chance that a sample of 27 contains nobody who can distinguish would be small but significant -- around 6%.
Floccina, what awesome math can he do? I'm curious.
Although the guy we tested seemed to be able to tell the difference between Coke and Pepsi I would still be surprised to learn that many others could do the same. The guy we tested had strange habit and abilities. He drank a lot of coke and was able to list like 50 digits of pi and do some awesome math off the top of his head and yet was unable to hold a job.
Psychohistorian, subjects were told the payment (a $50 gift certificate) would go to the person who came closest to guessing the right answer, so that should motivate subjects to get it right rather than to say what the experimenter wanted to hear. (The paper doesn't tell whether the winner was actually paid!)
Based on the skeptical comments here, I looked for articles on how accurate people actually are at guessing Coke vs Pepsi. Apparently it is a widely assigned student project but I couldn't find much in the way of rigorous studies on it. The paper linked in the article says they did test people to see how they would do:
"we assumed that informing participants of the drinks' identities ahead of time would lead them to taste a difference between them - a difference that would not be apparent to those uninformed of the identities. Indeed, in a pilot test of this assumption, 85% of students in the condition informed of the drinks' identities (N = 27) believed they could have correctly distinguished between the drinks even if they had not been informed - a figure that differed substantially from the 50% accuracy rate actually obtained from a sample of uninformed participants"
So only a 50% accuracy rate among people who weren't told which was which.
Another paper I found was mostly about testing people in an MRI machine to see how their brains lit up while tasting the drinks, but also did some studies of drink preferences outside the machine as controls. These people were not asked to guess Coke vs Pepsi per se, but they indicated which drink they habitually preferred, and then were asked to choose the drink they liked best from tasting two unlabeled cups. They made them do the taste comparison 15 times. Figure 1D shows the results: among people who preferred Coke, they picked the Coke cup 7.5 times out of 15 on average; Pepsi drinkers picked the Coke 6.8 times out of 15. So there was a small difference in the expected direction, but it was not statistically significant. The bottom line is that close to 50% of people picked the wrong drink.
The confidence among posters here is therefore rather surprising. Either it represents overconfidence bias, or else the article has prompted a selection effect so that people who are unusually adept at perceiving these taste differences are motivated to comment.
This may be more of a "guessing the password" answer. When people are asked without the cash incentive, they offer an honest (though possibly biased) estimate. When they have the cash incentive, they guess the answer they think you want to hear. It coincides with the truth in this case, but that is largely coincidental.
I'd bet that if you predisposed them to some (arbitrary) hypothesis, say from the researchers, say, 37%, the cash incentive would cause their responses to cluster more tightly around that point than those without the cash incentive,and it would have nothing to do with the truth.
I once conducted a double blind taste test using coke and pepsi with my sister as the taster and someone else to do the blinding. She said she loved coke and hated pepsi, I predicted she would be unable to distinguish them, she was able to but she repeatedly thought the pepsi was the coke and vise versa. After the findings were reported she actually said something like "Well I like coke cause it's american." - I feel like that was the best thing I've ever done in terms of a clear example of the usefulness of modern scientific testing to regular people.
My business partner and I thought that a person could not tell the difference between Coke and Pepsi but a guy working in the next office bet us that he could. Money was wagered. To make it more difficult we included some store brands and RC cola, which he insisted he could easy tell where not coke or Pepsi. He had most difficulty distinguishing between RC and Pepsi but in the end he identified Coke and Pepsi correctly and the store brands and RC as not coke or Pepsi and we paid. Now I must say that I could never do that.
Way of topic but this is not the first bit of independent research that I have done, I was working managing a restaurant at the time of the big debate about government provided daycare and we had a good number of single moms working in the restaurant so asked them would they rather have free daycare for their children or $10, $20, $30, $40, $50 given to them per week. It turned out that most had arrangements with family or friends and would take less than $20/week above free day care. Some said that they would not use the day care at all even if it was provided free to them.
Hugo, I can't imagine any treatment would make rationality violations disappear. Getting better performance on judgment tasks is exactly what I'm talking about.
Don't dismiss the 50-50 result so quickly. I also find Coke and Pepsi to taste different. But most people, myself included, cannot distinguish between a *sip* of each, particularly when the beverages are cold. It's an easy test to do on a reasonable sample.
My issue with the cited study is more likely to be whether the results actually stem from an egocentric bias. I have not read the study but too much research shoehorns the findings into the authors' pet theories. I want to believe the higher level conclusion that people make better decisions when the decisions matter but the cited study does not seem like the first experiment that I'd design to test it.
but see:Camerer et Hogarth 1999 The Effects of Financial Incentives in Experiments A Review and Capital-Labor-Production Frameworkabstract:We review 74 experiments with no, low, or high performance-based financial incentives. The modal result is no effect on mean performance though variance is usually reduced by higher payment.. Higher incentive does improve performance often, typically judgment tasks that are responsive to better effort. Incentives also reduce ‘‘presentation’’ effects e.g., generosity and risk-seeking.. Incentive effects are comparable to effects of other variables, particularly ‘‘cognitive capital’’ and task ‘‘production’’ demands,and interact with those variables, so a narrow-minded focus on incentives alone is misguided. We also note that no replicated study has made rationality violations disappear purely by raising incentives.
Let's repeat that last sentence:We also note that no replicated study has made rationality violations disappear purely by raising incentives.
Note, I am defying the data of these studies "Studies show that people's ability to do this is no better than chance", not the new one.
I actually think the ability of the whole population to do this is probably very little better than chance, but only because most people have never drunk either often enough to know the taste.
I defy the data of those studies.
This notion that you can't tell the difference is ridiculous, unless your subject population rarely drinks either. I've been served coke having ordered a pepsi, and I know it. It just doesn't taste the same.
I'd be willing to bet a lot of money on my ability to tell coke from pepsi in blind taste tests with a disinterested labeler, and I rarely drink cola anymore.
celeriac's results don't surprise me in the least. I would expect anyone who drank one or the other regularly and expressed a strong preference to be able to tell the difference pretty reliably (at least 80% chance). I expect I'd get pretty close to 100% under clean trial conditions (bottled not fountain, served in clean glass glasses).
Know of any research that illuminates subjectively experienced thought processes involved in ego-centric mode and analytical mode? Protocol analysis of some sort for instance?
The New Scientist link goes to this abstract:
"Take this study designed to see how we judge trustworthiness. Lisa DeBruine at the University of Aberdeen in the UK had volunteers play an online game in which they had to decide whether to trust another player to split a sum of money. All they had to go on was a picture. Volunteers were shown photos of students at another university, and also saw photos that DeBruine had manipulated electronically to incorporate aspects of the volunteers' own faces. The manipulations were subtle enough for the volunteers not to notice them but the effect was remarkable nevertheless, with players far more likely to trust the face."