In reading the comments on my variance-induced test bias post, I was reminded of a big bias loophole in social science: judging when an analysis is complete "enough." We usually have some status quo policies, and some analyses relevant to those policies. Each analysis tends to favor some possible policies relative to others, but alas most every analysis is
Robin, it doesn't seem to me like you're setting "prediction markets" against the strongest alternatives if you put up a version of "consensus of experts" that you admit is a weakened representation: "we follow the analyses preferred by the most prestigious academics available", because you claim "academic experts are chosen primarily for producing impressive analyses, and impressiveness is only somewhat related to accuracy and completeness".
For example, how about the consensus of experts chosen primarily for producing analysis that is most accurate?
Robin, excellent paper. Your data turns my point around. This is a great way to make money off of people with incorrect beliefs--what better way to drain the coffers of the biased and foolish?
There is still the problem that if a prediction market is linked to action, the prediction can't be something that would be altered by that action.
Of course if one of the outcomes of the prediction could affect the market itself, there's another problem. There would be no point in better on global disaster, with no way to realize the benefits of winning.
I imagine that these issues have come up in other papers so I'll stop there.
number 5 seems to allow for most options, since individuals deciding for themselves could choose to hire represenatives, enact decision markets, follow the most detailed analysis, et c. in deciding what policy to enact.
i'm not sure what would be best regarding choices that require consensus. "vote on values, bet on beliefs" seems attractive, so represenatives and prediction markets would be the attractive decision tools--prediction markets would make use of various analyses out there in the world, presumably.
but to be honest, if the values voted on run counter to my own values, i'm disinclined to go with the consensus, and i suspect many, perhaps most people think along these lines. voting seems an attractive means compared to others of resolving conflict, but not always. 'vote when it doesn't seriously move us away from advancing my libertarian values, or when it will advance my libertarian values as an aim of government, bet on beliefs regarding how to best advance those libertarian values.' that might be my verbose motto.
Hopefully, academic experts are chosen primarily for producing impressive analyses, and impressiveness is only somewhat related to accuracy and completeness.
Allan, Typepad wanted on that spelling; yet another reason to dislike Typepad.
Gavin, we have lots of data and theory on such manipulation, suggesting it isn't a problem.
A prediction market that is known to drive a decision will fail. Proponents of one or both courses of action will flood it with money based on their beliefs, and consider it a "donation" if it fails.
Imagine a market set up to predict the potential effects of climate change, that is directly tied into the level of a carbon cap. It would quickly become a "vote with your wallet" poll. Many would view the expenditure as a donation rather than an investment in a prediction.
Sadly, this isn't too far from our actual political system, and at least the people who are right would see some benefit, rather than only the advertising companies. Still, ideological donations could weaken any prediction markets given decision force.
It seems to me that everything rational is recursive ultimately to the decision of experts. If it's not experts on the specific topic decided upon, than it's experts in decision markets, or an expert in the democratic process, or an expert in the degree to which each person is being allowed to decide for themself. So I think that de-exceptionalizes the closest thing you have to an expression of that, in rather warped form, as your #2.
I lean towards the consensus of subject matter experts, who are free to use 1-5, mediated by decision theory experts. I think such decisions should be subject to periodic review and new data comes in, and that where possible their should be regional experimentation to help produce such data (even to the extent of double blind trials and all that).
Incomplete Analysis
Robin, it doesn't seem to me like you're setting "prediction markets" against the strongest alternatives if you put up a version of "consensus of experts" that you admit is a weakened representation: "we follow the analyses preferred by the most prestigious academics available", because you claim "academic experts are chosen primarily for producing impressive analyses, and impressiveness is only somewhat related to accuracy and completeness".
For example, how about the consensus of experts chosen primarily for producing analysis that is most accurate?
Robin, excellent paper. Your data turns my point around. This is a great way to make money off of people with incorrect beliefs--what better way to drain the coffers of the biased and foolish?
There is still the problem that if a prediction market is linked to action, the prediction can't be something that would be altered by that action.
Of course if one of the outcomes of the prediction could affect the market itself, there's another problem. There would be no point in better on global disaster, with no way to realize the benefits of winning.
I imagine that these issues have come up in other papers so I'll stop there.
Nevermind, I thought that was something you had talked about earlier than the variance-induced bias post.
Could you link to your proposal on SAT scores?
number 5 seems to allow for most options, since individuals deciding for themselves could choose to hire represenatives, enact decision markets, follow the most detailed analysis, et c. in deciding what policy to enact.
i'm not sure what would be best regarding choices that require consensus. "vote on values, bet on beliefs" seems attractive, so represenatives and prediction markets would be the attractive decision tools--prediction markets would make use of various analyses out there in the world, presumably.
but to be honest, if the values voted on run counter to my own values, i'm disinclined to go with the consensus, and i suspect many, perhaps most people think along these lines. voting seems an attractive means compared to others of resolving conflict, but not always. 'vote when it doesn't seriously move us away from advancing my libertarian values, or when it will advance my libertarian values as an aim of government, bet on beliefs regarding how to best advance those libertarian values.' that might be my verbose motto.
Hopefully, academic experts are chosen primarily for producing impressive analyses, and impressiveness is only somewhat related to accuracy and completeness.
Allan, Typepad wanted on that spelling; yet another reason to dislike Typepad.
Gavin, we have lots of data and theory on such manipulation, suggesting it isn't a problem.
A prediction market that is known to drive a decision will fail. Proponents of one or both courses of action will flood it with money based on their beliefs, and consider it a "donation" if it fails.
Imagine a market set up to predict the potential effects of climate change, that is directly tied into the level of a carbon cap. It would quickly become a "vote with your wallet" poll. Many would view the expenditure as a donation rather than an investment in a prediction.
Sadly, this isn't too far from our actual political system, and at least the people who are right would see some benefit, rather than only the advertising companies. Still, ideological donations could weaken any prediction markets given decision force.
I hate to do anything as lame as correct spelling, but I think you mean "analyses" when you write "analyzes".
It seems to me that everything rational is recursive ultimately to the decision of experts. If it's not experts on the specific topic decided upon, than it's experts in decision markets, or an expert in the democratic process, or an expert in the degree to which each person is being allowed to decide for themself. So I think that de-exceptionalizes the closest thing you have to an expression of that, in rather warped form, as your #2.
I lean towards the consensus of subject matter experts, who are free to use 1-5, mediated by decision theory experts. I think such decisions should be subject to periodic review and new data comes in, and that where possible their should be regional experimentation to help produce such data (even to the extent of double blind trials and all that).