The latest Journal of Prediction Markets describes lab experiments comparing prediction markets to deliberation. They collected 24 groups of four people to forecast a survey on the popularity of five cell phone designs. Half the groups talked among themselves and then made a group forecast, with prizes going to the best groups. In the other half of the groups the individuals rotated trading with an automated market maker, with prizes going to the best individual forecasters. They found:
Further to Hal's point, might the increased variance of group prediction simply be due to risk preferences inadvertently created by the experiment's design? The groups are competing against 11 other deliberative groups while the individuals are competing against only 3 other traders. If it's something like a winner-take-all contest, wouldn't we expect more variance where there are more competing units?
Now if the increased variance of the deliberative groups was instead caused by internally unequal endowments of aggression and persuasive skills, while the traders' budgets were equal, isn't it the case that most markets feature (wildly) unequal trading budgets along with non-independent, feedback trading? If I remember correctly, your experiment assumed equal trading budgets and no feedback trading.
Robin, your quotation says that the market groups had lower variation but does not say whether they did better. The abstract does say that they did better: "Using an experimental setting, we find that information markets provide more accurate and less volatile forecasts than group deliberations."
Lemmus, there seems to be a lack of consensus on group versus individual performance. A widely cited paper by Cooke and Kernaghan says:
"An extensive amount of research has focused on the relative performance of groups versus individuals in problem-solving situations. The results of this research have been inconsistent. To some extent these inconsistent findings can be attributed to differences in the variables used to represent individual and group output and the methods employed to compare their performance. This research uses data from 61 groups (347 individuals) who completed a planning simulation to review, compare, and contrast alternative strategies (or 'scoring algorithms') for estimating the differences between group versus individual performance on problem-solving tasks. Although the alternative strategies produced different estimates of the amount of gain that can be attributed to group interaction, they generally supported the conclusion that groups outperform their individual members. These results are discussed in terms of research on group performance, the use of simulations for training, and the role of groups in organizational problem solving and task performance."
Jason, good point about variance and the prize incentives.
Further to Hal's point, might the increased variance of group prediction simply be due to risk preferences inadvertently created by the experiment's design? The groups are competing against 11 other deliberative groups while the individuals are competing against only 3 other traders. If it's something like a winner-take-all contest, wouldn't we expect more variance where there are more competing units?
Now if the increased variance of the deliberative groups was instead caused by internally unequal endowments of aggression and persuasive skills, while the traders' budgets were equal, isn't it the case that most markets feature (wildly) unequal trading budgets along with non-independent, feedback trading? If I remember correctly, your experiment assumed equal trading budgets and no feedback trading.
Robin, your quotation says that the market groups had lower variation but does not say whether they did better. The abstract does say that they did better: "Using an experimental setting, we find that information markets provide more accurate and less volatile forecasts than group deliberations."
Lemmus, there seems to be a lack of consensus on group versus individual performance. A widely cited paper by Cooke and Kernaghan says:
"An extensive amount of research has focused on the relative performance of groups versus individuals in problem-solving situations. The results of this research have been inconsistent. To some extent these inconsistent findings can be attributed to differences in the variables used to represent individual and group output and the methods employed to compare their performance. This research uses data from 61 groups (347 individuals) who completed a planning simulation to review, compare, and contrast alternative strategies (or 'scoring algorithms') for estimating the differences between group versus individual performance on problem-solving tasks. Although the alternative strategies produced different estimates of the amount of gain that can be attributed to group interaction, they generally supported the conclusion that groups outperform their individual members. These results are discussed in terms of research on group performance, the use of simulations for training, and the role of groups in organizational problem solving and task performance."
Isn't there a voluminous literature in social psychology showing that groups do a much worse job at decision making than individuals?
Isn't there a voluminous literature in social psychology showing that groups do a much worse job at decision making than individuals?