A Mencius Moldbug has written a confused and rambling 7400 word critique of futarchy. But since Mencius seems to have passion and potential, let me try to communicate. Most readers may prefer to skip this post; it will get tedious.
Hanson has no problem with "shopping" for legal systems. Nor does he believe in an "objective" value function, he stated before his position on metaethics is moral anti-realism (same as mine) though he tried to work out a way to say that the moral realist consensus of philosophers wasn't a disagreement. Michael Abramowitz made the same point as you about the possible errors behind defining GDP+ when arguing for Predictocracy (bet on laws we will retroactively approve of, which I think a bad idea since I don't approve of many old but popular laws!) against Futarchy. I think his reason for voting on values is that he wants to "make a deal" with people who currently favor democracy.
Do excuse the necro-posting, but I was startled to discover no-one's even touched on what I feel is the weakest link in the futarchy proposal.
First, let me just say that prediction/futures markets are awesome, and Hanson's motivation -- that thinking up effective and economic public policies ought to be profitable too is a laudable one.
With that said, the whole system falls at the first hurdle, when we are required to vote on the way of evaluating the outcome of public policies. Not only does it assume that 1. there is some objective "value function" for evaluating public policy, but it assumes that -- 2. whilst democratic vote is deemed a poor way of working out the outcomes of public policy, it is a good way to work out what those outcomes should be
Assume, for instance, that I am a fundementalist Mormon. For me, an important aspect of public policy is that it minimises drug use and permits bigamy.
I am a minority in the US, so my democratic votes are not going to have that much effect on the value-function. Whether the decision markets work or don't, I'm going to be pretty dissatisfied with the outcome.
However, if we abandon the idea of a universal value-function, we may also abandon the controversial decision market for predicting public policy outcomes. I "shop" for a legislature in the same way I shop for baked beans; by their outcomes. If a legislature's public policy provides poor outcomes -- intended or not -- I will choose another.
The exact method of "shopping" for a legislature may vary. Moldbug's Patchwork, Friedman's PDAs, or even the old-school Panarchy are all adequate solutions.
This, in my opinion, is a far better way to make inventing good policy profitable.
TGGP, when I try to imagine a realistic way in which "fire-the-CEO" markets become effective, what I imagine is that as the evidence mounts for the value of those markets, boards of directors that fail to follow the results of the market suffer gradually increasing risks of being sued for breach of fiduciary responsibility and/or voted out, and the quality of evidence they need to present to defend themselves against such actions keeps increasing.
Also, I expect it will take some skill to read market prices and determine whether they are sending a signal or showing only noise. I expect boards of directors to initially have much discretion in making that distinction, and that as evidence grows the range of prices over which they can plausibly use this discretion will shrink, and will be gradually codified into rules.
I have trouble identifying a specific point in these trends when the process becomes mechanical. Are you imagining something different, such as some software that sends out "you are fired" letters being turned on?
TGGP, yes a manipulator's payoff is the improvement in the decision induced by the distorted market estimates, minus trading losses from such distortions. And yes part of the reason to put a mechanism directly in control is to avoid losses from overconfident human corrections.
Peter, yes it seems that the electoral college slowly and gradually became less important, so that national elections slowly became more mechanically in control of who is president. So there are degrees of direct control, and mechanical processes can slowly and gradually be placed in charge.
Peter, in Supercrunchers it is explained that experts assisted by prediction algorithms outperform experts by themselves. However, they underperform prediction algorithms alone. This is because people are sure the algorithm is wrong in certain instances and ignore its advice. In the reverse case where the algorithm has final decision power and is aware of an expert's prediction, it outperforms the algorithm by itself. Hanson has said he wants to create a "fire-the-CEO" market, which is definitely a case where human discretion could lead to incorrect overriding. So I do think something important happens when it is made "mechanical" rather than merely advisory.
To those who imagine something important happening when the market is "switched" from advisory to mechanically binding, think about the way a U.S. president is elected. Early presidential elections clearly didn't mechanically translate popular votes into a certification of a victor. Has the electoral process become mechanical now? If so, when did it become mechanical? If not, how many people are actively working to make it more mechanical and do you have any reason to expect futarchists to expend effort making futarchy more mechanical than the electoral college currently is?
Let's a imagine a policy in a subsidy-free decision market whose effects include transferring $X (an amount I already have or could raise) to me. I am indifferent about the other effects. I will then always be willing to bid up to $X (the point at which I am indifferent) in order to get the policy enacted. The guard against this is that other speculators will bid against me. If they are successful in moving the price back, the policy will not be enacted. Because my bet was conditional on the policy being enacted, my situation would then be unchanged. If the policy does get enacted then even if it fails to achieve the results I bet that it would, I still profit by X minus however much I bid (on the off-chance it does produce the results I bet on I receive X plus my winnings). Is all that correct?
your papers regarding the manipulator problem cover prediction markets where the rewards to the manipulators are on a similar order of magnitude as the stakes of the market players. In a futarchy model, this is likely to not be the case.
As an example, imagine that the State of California has set up a decision market to decide whether to raise certain taxes or to cut certain programs/benefits/salaries to balance its budget in the forthcoming fiscal year. The payoff to the decision market players is based on whether the budget balances while avoiding certain disfavored results.
Public-employee unions have, in general, a very large incentive to push the contract towards the "raise taxes+maintain spending" outcome. Very few businesses or individual taxpayers have similarly large incentives in the opposite direction, and based on prior history, it's unlikely that a coalition would form which was willing to spend the amount of money that the unions would spend.
To some extent, California's initiative process provides a modest test of manipulation incentives, though there are some significant differences between propagandizing on initiatives and participating in a market.
Whether they're "accurate" depends in large part on definitions of accurate, and who controls those definitions. They'll be different in decision markets (where accuracy depends on policy outcomes) than prediction markets (which simply predict, or not, what outcome will occur).
We face this today and I do not believe we have *truly* objective criterion for evaluating whether the implementation of a policy on any subject of import the least bit controvercial was accurately predicted. That is to say, we already have a lot of data on policy implementations and outcomes, and it's much harder to agree on whether their effects were accurately predicted in advance than with a market that simply predicts whether A or B will happen ("Will Barak Obama's Stimulus Package pass and be signed into law" is a much simpler question than "what will the effect of Stimulous Package A be, will it be better than Stimulous Package B, and will either be better than doing nothing, and over what time frame, and with or without other ripple effects" - CBO's predictions, for example, are considerabily different than the White House's).
In the abstract, people propose Policy A, others say Policy B will be better, some say the status quo is better than either. The advocates of Policy A say it will cost $100 Billion Woolongs and create 1.21 gigajobs. Critics say that it will cost four times that much in the end and produce half as many jobs.
Policy A is enacted, it ends up costing several multiples of what was predicted and net job growth is undetectable. Advocates say it was because it was underfunded at the beginning, and that other factors impacted it, and given how the economy performed, conditions would have been worse without it: It saved jobs. Critics say their criticisms were accurate and that the policy made things worse ultimately, particularly because of unintended consequences (Brawndo consumption went up 400%, causing crop failures and an increase in illiteracy and out of wedlock childbirth, and these things aren't included in the advocate's data regarding whether the policy outcome was good or bad). Adocate's rejoider is that well B, C, and D happened in the interval as well, and without Policy A in place, those things would have excacerbated the situation, so A really saved us from catastrophy.
This is much more open to manipulation than a prediction market in that sense alone.
Also, I think Hanson is well too dismissive of the ability of "Bears" to manipulate outcomes. In a world where George Soros & friends could cause a run on the Pound, it's not hard to foresee that a subset of interested parties who will benefit disproportionately from the implementation of a policy will have more interest in investing enough to make sure it happens than everyone else. Will "Wolves" follow that money, because it is more likely to occur (and thus pay out), or will they vote with the "sheep" (those whose direct interest in the policy choice is less strong, and indeed who may pay the costs and receve no benefit)?
"Smart Money" is more likely to go with the "Bears" on this, in my opinion. Their market behavior will be based on an evaluation of what will "win" rather than which will be the best policy to implement, and everyone else in the market will then be "sheep" (AKA the "marks").
For some reason a lot of the exchanges on "Wolves" vs. "Sheep" reminded me of Phil Helmuth going on rants about "idiots" who have offsuit Queen-Ten and raise. Phil's "idiots" win hands, get to final tables, even win tournaments, despite the fact that though over time - if your time horizon is long enough - the pros do much better than "dead money". When interest and passion is high, a lot of sheep enter the market who otherwise might not be there - and this will make the outcomes different than in the smaller-scale running experiments Hanson foresees as testing and building the case for Decision Markets.
But in this situation the way the sheep will be separated from their money by the wolves is not directly tied to which policy outcome was accurately predicted to be preferable. The decision market will be dominated by those "Wolves" who are most skilled in knowing which decision will dominate, not which one is good policy and which one(s) are worse, and by players with an interest in the outcome (Bears) regardless of it's overall impact (the "I'm All Right, Jack" set). The wolves will prosper in the market from investing correctly in which policy alternative will win, not which one will have the most benefitial impact.
It "may" (to use Professor Hanson's word) be that those don't correlate at all. They "may" even be opposed. I'm not sure I'd bet my life that they will be alligned. I agree with Moldbug that there is no good way to test it - I have little confidence that "experimenting" in a small scale on inconsequential policy decisions won't produce the same outcomes that going big on major choices will. Wolf money will tend to follow, rather than counterballance, Bear money, and everyone else by definition will become Sheep and get shorn.
Well! I feel it's only appropriate to give Professor Hanson the last word.
Conveniently, I finished the aforementioned paper - and discovered that it concludes with an admirably-motivated, and quite plausibly complete, list of all its spherical cows. (In case it isn't obvious, my objection above was to the last-named cow.) If this doesn't place Professor Hanson in the 99th percentile of academic honesty, I myself am a spherical cow:
Of course the fact that we have a particular model illustrating these results hardly implies that these results always hold in every context. Our model assumes risk-neutrality, normally distributed values and signal errors, interior choices of information quantity, only quantal-response-type irrationality, no meta-signals about the signals of other agents, no transaction costs of trading, no budget constraints, and a single rational manipulator with quadratic manipulation preferences and a commonly known strength of desire to manipulate. However convenient these assumptions may have been for solving the model, one can reasonably question the empirical relevance of models based on them.
Note that if the decision is welfare-diminishing compared to the status quo, there will be a group of people losing more money than the bears gain. They would have a strong incentive to buy the decision back, wouldn't they?
They have the same incentive to buy the decision back as the consumers do to buy an open market in foreign sugar. Unfortunately, corn producers and sugar producers have stronger incentive to buy the market closed.
Note that if the decision is welfare-diminishing compared to the status quo, there will be a group of people losing more money than the bears gain. They would have a strong incentive to buy the decision back, wouldn't they?
I think that's the cleverest argument on this whole thread! Note, however, that this would create a kind of policy-control contest very familiar to, say, the citizens of the late Roman Republic. I'm not sure it passes Schneier's test. Well, okay, I'm sure it doesn't. On the other hand, that doesn't mean it wouldn't be an improvement on Washington's present policy process...
All I can say is that if everyone approached prediction markets with the same skepticism as you, they would be truly useful tools. If they weren't basically illegal, that is.
There is no philosopher's stone. Good government is government by good people. In essence, an iterated prediction market is just a very clever way to employ good people, in a very unusual organizational structure which is unusually tolerant to the presence of bad people in the population. (Not to mention unusually inexpensive from the management perspective.) Treat the tool as what it is, and you can't go wrong. Treat it as a philosopher's stone...
Reply to Moldbug
Ah, that makes perfect sense. I'll try to track down his post on meta-ethics, thanks.
Hanson has no problem with "shopping" for legal systems. Nor does he believe in an "objective" value function, he stated before his position on metaethics is moral anti-realism (same as mine) though he tried to work out a way to say that the moral realist consensus of philosophers wasn't a disagreement. Michael Abramowitz made the same point as you about the possible errors behind defining GDP+ when arguing for Predictocracy (bet on laws we will retroactively approve of, which I think a bad idea since I don't approve of many old but popular laws!) against Futarchy. I think his reason for voting on values is that he wants to "make a deal" with people who currently favor democracy.
Do excuse the necro-posting, but I was startled to discover no-one's even touched on what I feel is the weakest link in the futarchy proposal.
First, let me just say that prediction/futures markets are awesome, and Hanson's motivation -- that thinking up effective and economic public policies ought to be profitable too is a laudable one.
With that said, the whole system falls at the first hurdle, when we are required to vote on the way of evaluating the outcome of public policies. Not only does it assume that 1. there is some objective "value function" for evaluating public policy, but it assumes that -- 2. whilst democratic vote is deemed a poor way of working out the outcomes of public policy, it is a good way to work out what those outcomes should be
Assume, for instance, that I am a fundementalist Mormon. For me, an important aspect of public policy is that it minimises drug use and permits bigamy.
I am a minority in the US, so my democratic votes are not going to have that much effect on the value-function. Whether the decision markets work or don't, I'm going to be pretty dissatisfied with the outcome.
However, if we abandon the idea of a universal value-function, we may also abandon the controversial decision market for predicting public policy outcomes. I "shop" for a legislature in the same way I shop for baked beans; by their outcomes. If a legislature's public policy provides poor outcomes -- intended or not -- I will choose another.
The exact method of "shopping" for a legislature may vary. Moldbug's Patchwork, Friedman's PDAs, or even the old-school Panarchy are all adequate solutions.
This, in my opinion, is a far better way to make inventing good policy profitable.
TGGP, when I try to imagine a realistic way in which "fire-the-CEO" markets become effective, what I imagine is that as the evidence mounts for the value of those markets, boards of directors that fail to follow the results of the market suffer gradually increasing risks of being sued for breach of fiduciary responsibility and/or voted out, and the quality of evidence they need to present to defend themselves against such actions keeps increasing.
Also, I expect it will take some skill to read market prices and determine whether they are sending a signal or showing only noise. I expect boards of directors to initially have much discretion in making that distinction, and that as evidence grows the range of prices over which they can plausibly use this discretion will shrink, and will be gradually codified into rules.
I have trouble identifying a specific point in these trends when the process becomes mechanical. Are you imagining something different, such as some software that sends out "you are fired" letters being turned on?
TGGP, yes a manipulator's payoff is the improvement in the decision induced by the distorted market estimates, minus trading losses from such distortions. And yes part of the reason to put a mechanism directly in control is to avoid losses from overconfident human corrections.
Peter, yes it seems that the electoral college slowly and gradually became less important, so that national elections slowly became more mechanically in control of who is president. So there are degrees of direct control, and mechanical processes can slowly and gradually be placed in charge.
Peter, in Supercrunchers it is explained that experts assisted by prediction algorithms outperform experts by themselves. However, they underperform prediction algorithms alone. This is because people are sure the algorithm is wrong in certain instances and ignore its advice. In the reverse case where the algorithm has final decision power and is aware of an expert's prediction, it outperforms the algorithm by itself. Hanson has said he wants to create a "fire-the-CEO" market, which is definitely a case where human discretion could lead to incorrect overriding. So I do think something important happens when it is made "mechanical" rather than merely advisory.
To those who imagine something important happening when the market is "switched" from advisory to mechanically binding, think about the way a U.S. president is elected. Early presidential elections clearly didn't mechanically translate popular votes into a certification of a victor. Has the electoral process become mechanical now? If so, when did it become mechanical? If not, how many people are actively working to make it more mechanical and do you have any reason to expect futarchists to expend effort making futarchy more mechanical than the electoral college currently is?
Let's a imagine a policy in a subsidy-free decision market whose effects include transferring $X (an amount I already have or could raise) to me. I am indifferent about the other effects. I will then always be willing to bid up to $X (the point at which I am indifferent) in order to get the policy enacted. The guard against this is that other speculators will bid against me. If they are successful in moving the price back, the policy will not be enacted. Because my bet was conditional on the policy being enacted, my situation would then be unchanged. If the policy does get enacted then even if it fails to achieve the results I bet that it would, I still profit by X minus however much I bid (on the off-chance it does produce the results I bet on I receive X plus my winnings). Is all that correct?
Anthony, you misread the models and experiments. Market speculators collectively have far deeper pockets than the California public employees union.
Robin -
your papers regarding the manipulator problem cover prediction markets where the rewards to the manipulators are on a similar order of magnitude as the stakes of the market players. In a futarchy model, this is likely to not be the case.
As an example, imagine that the State of California has set up a decision market to decide whether to raise certain taxes or to cut certain programs/benefits/salaries to balance its budget in the forthcoming fiscal year. The payoff to the decision market players is based on whether the budget balances while avoiding certain disfavored results.
Public-employee unions have, in general, a very large incentive to push the contract towards the "raise taxes+maintain spending" outcome. Very few businesses or individual taxpayers have similarly large incentives in the opposite direction, and based on prior history, it's unlikely that a coalition would form which was willing to spend the amount of money that the unions would spend.
To some extent, California's initiative process provides a modest test of manipulation incentives, though there are some significant differences between propagandizing on initiatives and participating in a market.
Whether they're "accurate" depends in large part on definitions of accurate, and who controls those definitions. They'll be different in decision markets (where accuracy depends on policy outcomes) than prediction markets (which simply predict, or not, what outcome will occur).
We face this today and I do not believe we have *truly* objective criterion for evaluating whether the implementation of a policy on any subject of import the least bit controvercial was accurately predicted. That is to say, we already have a lot of data on policy implementations and outcomes, and it's much harder to agree on whether their effects were accurately predicted in advance than with a market that simply predicts whether A or B will happen ("Will Barak Obama's Stimulus Package pass and be signed into law" is a much simpler question than "what will the effect of Stimulous Package A be, will it be better than Stimulous Package B, and will either be better than doing nothing, and over what time frame, and with or without other ripple effects" - CBO's predictions, for example, are considerabily different than the White House's).
In the abstract, people propose Policy A, others say Policy B will be better, some say the status quo is better than either. The advocates of Policy A say it will cost $100 Billion Woolongs and create 1.21 gigajobs. Critics say that it will cost four times that much in the end and produce half as many jobs.
Policy A is enacted, it ends up costing several multiples of what was predicted and net job growth is undetectable. Advocates say it was because it was underfunded at the beginning, and that other factors impacted it, and given how the economy performed, conditions would have been worse without it: It saved jobs. Critics say their criticisms were accurate and that the policy made things worse ultimately, particularly because of unintended consequences (Brawndo consumption went up 400%, causing crop failures and an increase in illiteracy and out of wedlock childbirth, and these things aren't included in the advocate's data regarding whether the policy outcome was good or bad). Adocate's rejoider is that well B, C, and D happened in the interval as well, and without Policy A in place, those things would have excacerbated the situation, so A really saved us from catastrophy.
This is much more open to manipulation than a prediction market in that sense alone.
Also, I think Hanson is well too dismissive of the ability of "Bears" to manipulate outcomes. In a world where George Soros & friends could cause a run on the Pound, it's not hard to foresee that a subset of interested parties who will benefit disproportionately from the implementation of a policy will have more interest in investing enough to make sure it happens than everyone else. Will "Wolves" follow that money, because it is more likely to occur (and thus pay out), or will they vote with the "sheep" (those whose direct interest in the policy choice is less strong, and indeed who may pay the costs and receve no benefit)?
"Smart Money" is more likely to go with the "Bears" on this, in my opinion. Their market behavior will be based on an evaluation of what will "win" rather than which will be the best policy to implement, and everyone else in the market will then be "sheep" (AKA the "marks").
For some reason a lot of the exchanges on "Wolves" vs. "Sheep" reminded me of Phil Helmuth going on rants about "idiots" who have offsuit Queen-Ten and raise. Phil's "idiots" win hands, get to final tables, even win tournaments, despite the fact that though over time - if your time horizon is long enough - the pros do much better than "dead money". When interest and passion is high, a lot of sheep enter the market who otherwise might not be there - and this will make the outcomes different than in the smaller-scale running experiments Hanson foresees as testing and building the case for Decision Markets.
But in this situation the way the sheep will be separated from their money by the wolves is not directly tied to which policy outcome was accurately predicted to be preferable. The decision market will be dominated by those "Wolves" who are most skilled in knowing which decision will dominate, not which one is good policy and which one(s) are worse, and by players with an interest in the outcome (Bears) regardless of it's overall impact (the "I'm All Right, Jack" set). The wolves will prosper in the market from investing correctly in which policy alternative will win, not which one will have the most benefitial impact.
It "may" (to use Professor Hanson's word) be that those don't correlate at all. They "may" even be opposed. I'm not sure I'd bet my life that they will be alligned. I agree with Moldbug that there is no good way to test it - I have little confidence that "experimenting" in a small scale on inconsequential policy decisions won't produce the same outcomes that going big on major choices will. Wolf money will tend to follow, rather than counterballance, Bear money, and everyone else by definition will become Sheep and get shorn.
So in other words, no change.
Well! I feel it's only appropriate to give Professor Hanson the last word.
Conveniently, I finished the aforementioned paper - and discovered that it concludes with an admirably-motivated, and quite plausibly complete, list of all its spherical cows. (In case it isn't obvious, my objection above was to the last-named cow.) If this doesn't place Professor Hanson in the 99th percentile of academic honesty, I myself am a spherical cow:
Of course the fact that we have a particular model illustrating these results hardly implies that these results always hold in every context. Our model assumes risk-neutrality, normally distributed values and signal errors, interior choices of information quantity, only quantal-response-type irrationality, no meta-signals about the signals of other agents, no transaction costs of trading, no budget constraints, and a single rational manipulator with quadratic manipulation preferences and a commonly known strength of desire to manipulate. However convenient these assumptions may have been for solving the model, one can reasonably question the empirical relevance of models based on them.
The prosecution rests - and calls for leniency.
Note that if the decision is welfare-diminishing compared to the status quo, there will be a group of people losing more money than the bears gain. They would have a strong incentive to buy the decision back, wouldn't they?
They have the same incentive to buy the decision back as the consumers do to buy an open market in foreign sugar. Unfortunately, corn producers and sugar producers have stronger incentive to buy the market closed.
Concentrated benefits, diffuse costs.
rafal,
Note that if the decision is welfare-diminishing compared to the status quo, there will be a group of people losing more money than the bears gain. They would have a strong incentive to buy the decision back, wouldn't they?
I think that's the cleverest argument on this whole thread! Note, however, that this would create a kind of policy-control contest very familiar to, say, the citizens of the late Roman Republic. I'm not sure it passes Schneier's test. Well, okay, I'm sure it doesn't. On the other hand, that doesn't mean it wouldn't be an improvement on Washington's present policy process...
Moldbug, you do not understand what you read. Our conversation is over.
Yvain,
All I can say is that if everyone approached prediction markets with the same skepticism as you, they would be truly useful tools. If they weren't basically illegal, that is.
There is no philosopher's stone. Good government is government by good people. In essence, an iterated prediction market is just a very clever way to employ good people, in a very unusual organizational structure which is unusually tolerant to the presence of bad people in the population. (Not to mention unusually inexpensive from the management perspective.) Treat the tool as what it is, and you can't go wrong. Treat it as a philosopher's stone...