Score Your Beliefs
A new Games and Economic Behavior paper (ungated here) shows that just asking folks to estimate the chances of events, where they expect to be scored later for accuracy, induces more accurate beliefs about those events, but that what folks believe and say can still consistently diverge:
Belief elicitation in game experiments may be problematic if it changes game play. We experimentally verify that belief elicitation can alter paths of play in a two-player repeated asymmetric matching pennies game. Importantly, this effect occurs only during early periods and only for players with strongly asymmetric payoffs, consistent with a cognitive/affective effect on priors that may serve as a substitute for experience. These effects occur with a common scoring rule elicitation procedure, but not with simpler (unmotivated) statements of expected choices of opponents. Scoring rule belief elicitation improves the goodness of fit of structural models of belief learning, and prior beliefs implied by such models are both stronger and more realistic when beliefs are elicited than when they are not. We also find that “inferred beliefs” (beliefs estimated from past observed actions of opponents) can predict observed actions better than the “stated beliefs” from scoring rule belief elicitation.
Yet more evidence that we should try to get into the habit of collecting track records about our beliefs.
Matt, very cool! :)
Try http://pbook.trike.com.au/ as a way of keeping track.(We're still in closed beta, but you're all close friends here. We'll go live at predictionbook.com when we're ready. Any history you generate before we go live will be preserved.)
Notes & disclaimers:<ul><li>If this app is wildly successful, I may make money from it - I hope you'll agree that this isn't spam because I'm an avid reader of OB, we wrote this app after reading OB and related material convinced me that it should exist, and I genuinely believe that some of you will like it.</li><li>We're still in "closed" beta, and there are some rough edges still to be worked off (in particular, if you're using an Internet Explorer, consider using a good browser instead - we're still fighting with IE and don't expect to be finished for at least a couple of weeks).</li><li>We're very interested in your feedback - please use the feedback tab on the right of the site.</li></ul>