Graeme Wood on Futarchy

At the end of his article on the deaths of Intrade and its founder John Delaney, Graeme Wood considers futarchy:

It’s perhaps no great surprise that we haven’t embraced Hanson’s “futarchy.” Our current political system resists dramatic change, and has resisted it for 237 years. More traditional modes of prediction have proved astonishingly bad, yet they continue to run our economic and political worlds, often straight into the ground. Bubbles do occur, and we can all point to examples of markets getting blindsided. But if prediction markets are on balance more accurate and unbiased, they should still be an attractive policy tool, rather than a discarded idea tainted with the odor of unseemliness. As Hanson asks, “Who wouldn’t want a more accurate source?”

Maybe most people. What motivates us to vote, opine, and prognosticate is often not the desire for efficacy or accuracy in worldly affairs—the things that prediction markets deliver—but instead the desire to send signals to each other about who we are. Humans remain intensely tribal. We choose groups to associate with, and we try hard to show everybody which groups we belong to. We don’t join the Tea Party because we have exhaustively studied and rejected monetarism, and we don’t pay extra for organic food because we have made a careful cost-benefit analysis based on research about its relative safety. We do these things because doing so says something that we want to convey to others. Nor does the accuracy of our favorite talking heads matter that much to us. More than we like accuracy, we like listening to talkers on our side, and identifying them as being on our team—the right team.

“We continue to have consistent results and evidence that markets are accurate,” Hanson says. “If the question is, ‘Do these things predict well?,’ we have an answer: They do. But that story has to be put up against the idea that people never really wanted more accurate sources.”

On this theory, the techno-libertarian enthusiasts got the technology right, and the humanity wrong. Whenever John Delaney showed up on CNBC, hawking his Intrade numbers and describing them as the most accurate and impartial around, he was also selling a future that people fundamentally weren’t interested in buying. (more)

I don’t much disagree — I raised these issues with Wood when he interviewed me. As usual, our hopes for idealistic outcomes mostly depend on finding ways to shame people into actually supporting what they pretend to support, by making the difference too obvious to ignore.

More specifically, I hope prediction markets within firms may someday gain a status like cost accounting today. In a world were no one else did cost accounting, proposing that your firm do it would basically suggest that someone was stealing there. Which would look bad. But in a world where everyone else does cost accounting, suggesting that your firm not do it would suggest that you want to steal from it. Which also looks bad.

Similarly, in a world where few other firms use prediction markets, suggesting that your firm use them on your project suggests that your project has an unusual problem in getting people to tell the truth about it via the usual channels. Which looks bad. But in a world where most firms use prediction markets on most projects, suggesting that your project not use prediction markets would suggest you want to hide something. That is, you don’t want a market to predict if your project will make its deadline because you don’t want others to see that it won’t make the deadline. Which would look bad.

Once prediction markets were a standard accepted practice within firms, it would be much easier to convince people to use them in government as well.

GD Star Rating
a WordPress rating system
Tagged as: ,
Trackback URL:
  • IMASBA

    “We don’t join the Tea Party because we have exhaustively studied and rejected monetarism, and we don’t pay extra for organic food because we have made a careful cost-benefit analysis based on research about its relative safety.”

    … and maybe we don’t recommend futarchy because mythical meticulous research has shown them to be good for everybody (but because we want to signal we are good American free market-fetishists who love giving billionaires new ways to screw with democracy, and we can show how very unorthodox we are in our thinking)…

    • http://www.gwern.net/ gwern
      • IMASBA

        Oh, wow, prediction markets (that were not even manipulated because no one powerful used them to base deicsions on so no one had an incentive to manipulate, which they would have in a futarchy) actually aggregate the beliefs of their traders and they outperform election polls! Nate Silver, the NAS and the CBO better pack up now…

      • http://www.gwern.net/ gwern

        I’m sure the people who blew millions on the the Kerry and Romney contracts were just doing it on a lark and had ‘no incentive to manipulate’; it was just good clean fun.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        The possibility of manipulating prediction markets can be overcome by sufficient subsidy. (A few political markets have enough inherent interest.)

        But the same coalitions with the resources to manipulate the prediction markets have also always stood for downsizing government and lowering taxes. Hence, if the government had prediction markets, those markets would probably be under-subsidized, permitting their manipulation.

      • http://www.gwern.net/ gwern

        If the prediction markets were actually affecting anything, that sounds like it would attract even more inherent interest than they did before. Bit of a dilemma there – prediction markets without interest may be manipulable, but then why care about any manipulation and who would bother doing it…?

      • IMASBA

        It’s this false sense of balance (especially in the US) that needs to be gotten rid of. People are so used to false-balance from the media that they don’t recognize it for being false balance. The sad truth is that in reality there is just no way “inherent interests” of political progressives could ever compensate for the billions of dollars that political conservatives could pour into prediction markets and even if that was possible there are still plenty of other situations where prediction markets would be strongly manipulated by the side that has more money. I mean, seriously, let’s say there was a prediction market on the impact of fracking on ecology and public health in the United States where the US government based its long term policy on the position of that prediction market. In that case the Koch Brothers could easily pour $1 billion into the prediction market to reinforce the position that fracking won’t be so bad, because of this the government gives them more leeway to frack and they make $3 billion in extra profits, of course eventually they’ll lose their $1 billion when it turns out the market position was wrong, but the Koch Brothers won’t care, they’ve made $2 billion more than they would have otherwise. Contrary to what Faux News raving about George Soros and Warren Buffett want you to believe there aren’t really sufficient forces on the opposite side of the spectrum to counter-balance that $1 billion investment by the Koch Brothers (which will of course be channeled through some obscure way so it isn’t obvious the money came fromt them). Sure the rest of the world would frown upon this all but the US has a very long history of completely ignoring what the rest of the world thinks (healthcare, climate change, segregation, evolution, extreme economic libertarianism of which futarchy is a cancerous outgrowth).

      • IMASBA

        P.S.

        That’s all ignoring even the fact that prediction markets are inherently flawed compared to already flawed financial markets because financial markets at least influence future prices directly while prediction markets don’t (right or wrong the price of a house does strongly depend on what the market thought it was worth previously, there is a self-fulfilling element making financial markets better predictors than you would expect based on the aggregated knowledge of the traders, while the ecological and public health impact of fracking depends entirely on the laws of physics.)

      • http://www.gwern.net/ gwern

        > The sad truth is that in reality there is just no way “inherent
        interests” of political progressives could ever compensate for the
        billions of dollars that political conservatives could pour into
        prediction markets

        I wonder what all the corporate donors to the Democratic Party think they’re buying. Perhaps they, too, are just engaged in a lark.

      • IMASBA

        Since when does getting a Democrat elected mean progressive viewpoints will be carried out (banks donate to democrats to buy deregulation, Hollywood donates to get mroe stringent IP laws, neither of those are very progressive)? Even so in the last couple of American elections there clearly was one party that drew a lot more support from corporations and the rich and that side can easily double their donations while the other side simply can’t. With prediction markets this would all become a lot more frequent and ordinary progressive citizens just can’t keep up with that. But like I said, it’s just dangerous to assume a balance will always exist, no matter what the subject of the prediction market, not in the least because often the truth doesn’t lie in the middle at all (though people think it does, another consequence of false balance in the media). For example in a prediction market on climate change environmental and scientific advisory groups would have to outspend the Koch Brothers 992 to 1 (or more depending on the position of the rest of the market participants)!

      • http://www.gwern.net/ gwern

        Well, if you want to no-true-scotsman this discussion, I suppose you can. ‘No true progressive has ever been elected’ – but hey, on the bright side, if we switch to futarchy, progressives can hardly be worse off under that system than they are now.

      • IMASBA

        “Well, if you want to no-true-scotsman this discussion, I suppose you can. ‘No true progressive has ever been elected”

        That’s not what I’m saying: I’m saying progressive causes inherently have more trouble amassing funds because the rich and powerful became rich and powerful through the existing system and hence have an incentive to prevent change. On a deeper level I’m saying that even if progressive causes had no disadvantage or advantage that still would just leave a neutral position, which is fine if you’re a media outlet that wants to reach a large audience without offending anyone but it’s useless when you’re looking for the truth (the universe doesn’t care about American politicis or any other politics). It goes further than progressivism vs. conservatism too: any prediction market can be manipulated quite easily in a society where the richest 1000 people own more than the poorest 100 million and getting to the truth would often (when reality has a liberal bias) require the poor to outspend the rich by a large factor. And that’s just assuming prediction markets without manipulation are useful enough to overcome the negative effects of giving a false sense of security (prediction markets will be weaker than financial markets because there is no self-fulfilliling prophecy stuff going on and also complicated questions will lead to traders using only a handful of third party models that generate too little variance to provide an interesting market, meaning traders will revert back to their intuition, a prediction market on the weather would demonstrate that perfectly).

      • http://www.gwern.net/ gwern

        > manipulated quite easily in a society where the richest 1000 people own
        more than the poorest 100 million and getting to the truth would often
        (when reality has a liberal bias) require the poor to outspend the rich
        by a large factor.

        …and those 1000 people would very quickly become very poor if they were actually massively distorting an otherwise correctly-functioning prediction market. The failure mode of prediction markets is *not* ‘rich people are making bad estimates and outspending the rational participants’! The failure modes are more like ‘rich people are closing contracts the wrong way in their favor’.

        > complicated questions will lead to traders using only a handful of third
        party models that generate too little variance to provide an
        interesting market, meaning traders will revert back to their intuition,
        a prediction market on the weather would demonstrate that perfectly).

        You know, *actual* weather forecasters uses large ensembles of models with human input to gain predictive edges (I think Silver estimates the human addition as ~10% increase in accuracy over the current state-of-the-art models) – but oddly enough, they think this is a good thing and not a bad thing.

      • IMASBA

        “…and those 1000 people would very quickly become very poor if they were actually massively distorting an otherwise correctly-functioning prediction market.”

        Why? They can easily make up for their losses on the prediction markets by influencing government and corporate policies. And you know, there are parties that have even more money than the Koch Brothers, parties like the Chinese and Russian governments. All sorts of combinations are possible (it’s not just about American progressives vs. American conservatives): the Russian government could side with environmentalists and influence a prediction market so that the US government won’t allow drilling in the Gulf of Mexico, raising the value of Russian oil, just a 1% change in value would yield $20 billion per year for the Russian treasury, so pumping billions into that prediction market isn’t really a big deal for them.

      • http://www.gwern.net/ gwern

        If they can really make *so* much profit off the influenced policies that they can afford to pay off everyone in the world who are stampeding into the prediction markets to make free money, in what sense are their policies bad and the markets not functioning as designed?

      • IMASBA

        The whole working world? Don’t you mean working people in the same country who make more than minimum wage, feel confident and secure enough to gamble, want to go through the trouble of informing themselves about the subject, happen to put money in a prediction market that’s of interest to the rich and accounting for the fact that some of the public will take the same position as the rich? If that’s what you mean then I’m pretty sure that in the US the rich can certainly outspend everyone else in some markets and significantly distort the price in other markets.

        Are you seriously asking how anyone can say there’s something wrong with the world if there are rich people making a lot of money (as a percentage of the total economy)? Well, I don’t know, because life tends to suck if a small group of people tends to hold most of the power and wealth, or you know because there’s probably some pretty hefty rent-seeking going on?

      • http://www.gwern.net/ gwern

        If manipulation happens on as a regular and big a scale as in your fantasy scenario, no one needs to be well-informed because it will be obvious and everyone will be interested in joining on on the free money and so the poor will *especially* be interested. I think you underestimate how many poor people are able to hand over their money to complicated financial things – if poor Albanians can manage to put half the country’s GDP into Ponzi schemes, I’m sure that people can figure out how to deposit money into a prediction market and buy some shares…

        > If that’s what you mean then I’m pretty sure that in the US the rich can
        certainly outspend everyone else in some markets and significantly
        distort the price in other markets.

        Maybe you should look up actual distributions of wealth in populations.

      • IMASBA

        “If manipulation happens on as a regular and big a scale as in your fantasy scenario, no one needs to be well-informed because it will be obvious and everyone will be interested in joining on on the free money and so the poor will *especially* be interested.”

        There is no easy free money in it: while the trading is going on you never know if the manipulation (possibly by multiple parties) is going to succeed and the manipulation will surely be covert (there’s not going to be a cheque with the name “Koch” on it). Sure there will be traders who specialize in predicting if a market will be manipulated or not and placing bets contrary to the side most likely to manipulate the most, but that hasn’t stopped manipulation financial markets either. In most countries most people do not invest in the stock market (or only indirectly through government mandated pension funds), even though the expected returns are always positive if you spread out enough. Ordinary people just can’t stall $10.000 for 10 years on some gamble, and yet it’s amounts like that that people would need to invest in the right prediction markets to prevent manipulation from having an effect.

      • http://www.gwern.net/ gwern

        > There is no easy free money in it: while the trading is going on you
        never know if the manipulation (possibly by multiple parties) is going
        to succeed

        Given the conditionality, attempts to fight the manipulation aren’t much disadvantaged – if the condition falls through, no money changes hands, after all. What’s sauce for the goose is sauce for the gander…

        > and the manipulation will surely be covert (there’s not going
        to be a cheque with the name “Koch” on it).

        Doesn’t make the slightest difference. Do you think the Kerry and Romney manipulations came with signed cheques? Nevertheless, the Intrade forums and blogs erupted with accusations of manipulation.

        > but that hasn’t stopped manipulation financial markets either.

        The existing levels of manipulation like LIBOR hasn’t stopped the financial markets from being overall pretty efficient, either.

        > In most countries most people do not invest in the stock market (or only
        indirectly through government mandated pension funds), even though the
        expected returns are always positive if you spread out enough.

        They invest indirectly precisely because there are many other more convenient forms, and given the huge shift towards defined-contribution rather than defined-benefit retirement savings, peoples’ direct exposure to the stock market is going up all the time.

        > Ordinary people just can’t stall $10.000 for 10 years on some gamble,
        and yet it’s amounts like that that people would need to invest in the
        right prediction markets to prevent manipulation from having an effect.

        As your own examples point out, people do not need to lock up arbitrarily large sums for long periods on their own account: if that’s really an issue, they can invest in mutual-fund-like corporations whose shares they can buy and sell as they need to. Exercise a little imagination for once.

      • IMASBA

        “Given the conditionality, attempts to fight the manipulation aren’t much disadvantaged – if the condition falls through, no money changes hands, after all. What’s sauce for the goose is sauce for the gander…”

        Some markets will have conditionality, others won’t, in a conditional market the anti-manipulators do indeed have less of a disadvantage, but the manipulators won’t lose money if their manipulation fails, so it’s just neutral to them.

        “Doesn’t make the slightest difference. Do you think the Kerry and Romney manipulations came with signed cheques? Nevertheless, the Intrade forums and blogs erupted with accusations of manipulation.”

        Let me guess, two camps accused each other roughly equally and there was no way for people to know whether those were baseless accusations (no one was manipulating), half true (only one side was manipulating), true (both sides were manipulating or three quarters true (both sides were manipulating but the relative magnitudes were quite different from those of the accusations). Good luck coordinating an anti-manipulation campaign among all the noise. Disinformation is a b*tch.

        “The existing levels of manipulation like LIBOR hasn’t stopped the financial markets from being overall pretty efficient, either.”

        First of all the financial market is always right even when it’s not right (because it sets prices as well as trying to predict them: when the financial market assesses houses as more valuable than they really are then housing prices still go up), second, the financial market is very broad with many powerful competing interests, prediction markets will be much narrower and smaller (combined they may be very broad and large, but each individual market won’t be).

        “As your own examples point out, people do not need to lock up arbitrarily large sums for long periods on their own account: if that’s really an issue, they can invest in mutual-fund-like corporations”

        Good point, but then they’ll be left to the investment choices of those funds. In any case I think you really underestimate economic inequality: the richest 1% of the world own ~50% of the wealth, more if you look at liquid wealth and as I pointed out governments can also play the game.

      • IMASBA

        “You know, *actual* weather forecasters uses large ensembles of models with human input to gain predictive edges (I think Silver estimates the human addition as ~10% increase in accuracy over the current state-of-the-art models) – but oddly enough, they think this is a good thing and not a bad thing.”

        There are only a few models. Meteorologists the world over pretty much agree within strict boundaries which combination of models to use for a certain timespan for a certain location. Investors will go with this advice and, like meteorologists, will not disagree over whether a typhoon will hit China the day after tomorrow or not.

      • VV

        Unconditional prediction markets are expensive to manipulate.

        However, conditional prediction markets, which are the ones Hanson is talking about when he refers to “futarchy”, can be manipulated at no cost (other than transaction cost, if any), provided that you have a sufficient large capital.

      • http://www.gwern.net/ gwern

        Correct me if I’m wrong, but the conditional markets simply end in a no-trade if the conditional is not chosen but the trades are executed otherwise and pay out or do not pay out based on the pre-established metrics. So any manipulator can still be whacked by losses if their manipulation succeeds. That’s ‘no cost’ only if you think risk doesn’t exist.

      • VV

        The manipulator bets against the predicted perfomance of the policy that they don’t want to be selected. If the manipulation is successful, the policy is not selected, and the manipulator gets their money back. Hence, no cost.

        Of course, there is the risk that the manipulation fails, the policy is chosen and it turns out to be quite successful, thus the manipulator loses money. But if the manipulator is quite confident of their chances of success (because they they are the biggest player with a stake on the issue, for instance), then this can be an acceptable risk, considering the gains that can be obtained by manipulation.

      • http://www.gwern.net/ gwern

        So in other words, it’s only ‘no cost’ if you think taking on big risks like selling derivatives or shorting entails ‘no cost’. I don’t see why that’s a real problem for it then, especially as the more distorted the market gets and further from the optimal result, the more the manipulator is putting at risk if the manipulation ultimately fails.

      • VV

        Hedge funds won’t risk betting a large sum on the success of a policy unless they are very much confident of its success.

        The accuracy of any predictor is boundend by the fact that investing on increasing it runs in diminishing returns past a certain point. Moreover, predictors have an incentive to increase their accuracy only on predictions which have a significant chance of being selected: if they make an huge investment to made an accurate prediction on a certain policy and then the policy is not selected, the investment doesn’t pay off.

        Therefore relatively safe manipulation is certainly possible unless you are willing to make bizarre assumptions such as infinite liquidity.

        At worst, there can be multiple manipulators trying to pull the market in different directions, in which case the system becomes an all-pay auction. Certainly, that’s far from an unbiased prediction.

      • VV

        > all-pay auction

        Actually, it’s all-losers-pay, which means that if you know you are the biggest fish, you have an incentive to go for it.

      • http://www.gwern.net/ gwern

        > Hedge funds won’t risk betting a large sum on the success of a policy unless they are very much confident of its success.

        What? Hedge funds are famous for using extremely complex trading strategies which rely on finely-balanced probabilities and deploying huge sums to pick up tiny advantages, and this is most dramatically apparent in instances like LTCM. Where on earth is this claim coming from? Where is your evidence for this extraordinary claim?

        > The accuracy of any predictor is boundend by the fact that investing on
        increasing it runs in diminishing returns past a certain point.

        Which is why we have leverage and hedge funds can be levered by dozens of times – to amplify even small predictive advantages into huge profits.

        > Therefore relatively safe manipulation is certainly possible

        Again, it’s no more ‘relatively safe’ than shorting or selling insurance are: you may make small profits in the interim, but if you ever fail, you can lose everything.

      • VV

        What? Hedge funds are famous for using extremely complex trading
        strategies which rely on finely-balanced probabilities and deploying
        huge sums to pick up tiny advantages

        Yeah, that’s why hedge funds were moving so much money on Intrade. Oh wait, they weren’t.

        and this is most dramatically apparent in instances like LTCM

        Uh? According to Wikipedia that was a hedge fund that accumulated an enormous debt, failed, and was bailed out by the government. If that was your best example of market efficiency then you are not really helping your case.

      • IMASBA

        The betting typically won’t be on whether a policy will be enacted but on whether it should be enacted. For example a prediction market that predicts the outcome of a 10-year environmental report that the government isn’t willing to wait 10-years for and since they have faith in prediction markets they’re going with what the prediction market thinks the outcome of the report will be. Here an oil company could manipulate the prediction market and if they succeed reap the rewards for at least 10 years, of course after 10 years the report will find the prediction market was wrong but this will be swept under the carpet as a “statistical fluke”, the oil company now loses what they put into the prediction market but they made more than that in the 10 years that have passed, so the oil company made a profit by manipulating the prediction market. Mind you, the oil company may just have had to nudge the “clean” market position a little to get the desired result, that is certainly feasible with economic inequality and concentration of wealth being what they are.

      • IMASBA

        There’s always a cost if you manipulate the market away from the truth, but in futarchy there’s the opportunity to turn that into an investment by influencing government policy. The payoff can be huge in countries with high levels of economic inequality, such as the United States.

        Btw, why is no one asking why this idea is only discussed in the US, as so many extremely libertarian ideas? Do American academics simply not care if people as diverse as Chinese, Indians, Brazilians, Egyptians, Australians and Germans all think it’s a terrible idea? How can they just ignore that?

      • Curt Adams

        Regardless of whether the political prediction markets were manipulated, they’ve been absolutely atrocious in comparison to poll aggregators like Silver. Silver and Wang both nailed almost every state-level Presidential and Senate even though the prediction markets had lots of toss-ups. Certainly any claim that prediction markets are *necessarily* accurate or even good is resoundingly falsified.

        Are there any cases where prediction market have turned out to be the best available source of predictions? Is there any evidence at all that they can work sometimes in the real world?

      • http://www.gwern.net/ gwern

        Cite on those claims? As it happens, I did compare Intrade head to head with Silver and Wang to estimate their RMSE among other things (http://www.gwern.net/2012%20election%20predictions) and the outperformance was not that impressive, Silver and Wang were one of multiple pundits, in part they got lucky (Silver admitted as much, that part of the reason his tally looks so good is that some of his 51% calls happened to go his way), *and* for the 2012 elections, Intrade was hampered by the finance regs which barred American customers’ deposits – and the theory which predicts prediction markets work also predicts that when a prediction market is forced to exclude the majority of interested knowledgeable participants it will no longer be as accurate as, say, 2004.

      • IMASBA

        “in part they got lucky (Silver admitted as much, that part of the reason his tally looks so good is that some of his 51% calls happened to go his way)”

        Of course luck is involved, it’s statistics, not a crystal ball. Prediction markets also get lucky. The fact that a single man can outperform an un-manipulated prediction market should be humbling. If you want to get around US legal restrictions you could try looking at bettings made in the UK.

      • http://www.gwern.net/ gwern

        Erm, that was not what I meant. My point was that simple tallying methods like ‘Silver called all 50 states and Intrade only 48!’ is deeply misleading because the difference on a probability basis is minute: if Intrade puts a state’s Obama win probability at 49.9% and Silver at 50.1%, people will go around saying ‘Silver beat Intrade!’ even though it would take hundreds of such datapoints to start to approach any real confidence that the .2% difference in forecasts was real, much less in Silver’s favor. This is what Silver was talking about in his post-mortem when he noted that his performance was not as good as was being popularly claimed.

        > The fact that a single man can outperform an un-manipulated prediction market should be humbling.

        First, there were attempts to manipulate the 2012 Presidential markets. They didn’t succeed very well and were a colossal waste of money by the people who tried, but I am surprised that you can make that claim given that I already pointed out one well-established attempt. Second, Silver is not so much a single man as hundreds/thousands of polls. Third, I’ve already pointed out that the 2012 markets were badly crippled by most of the participants being banned from it (shortly followed by actual lawsuits by the American government); if the majority of the polls Silver relied on had been shut down by the government, I wonder how well his results would have been. And fourth, there’s selection effects here: just in my previously linked analysis, I included 9 different predictors – and that’s just the ones who provided me precise probabilities or estimates state-by-state! Because Silver’s margin was so small (and incidentally, why do you keep focusing on Silver? He was not the best forecaster in 2012! For example, he lost in several areas to Wang.), it isn’t much evidence of substantial improvement over anyone else.

      • Curt Adams

        First of all, this is mostly a sideline, making excuses for a prediction market failure. My big question is: are there *any* successes? Because if the story of prediction markets is nothing but making excuses for why other methods beat them, then they don’t work.

        There were markets abroad and they weren’t any better. The killer problem I see is that the prediction markets never put high confidence on any of the presidential outcomes (Obama-Romney was 60-40 in one market and 70-30 in another IIRC) while Silver and Wang had high confidences, which are supported by the fact that both are highly accurate. Likewise with Obama-McCain. Both times there was a clear and strong prediction, available based on public info, and the prediction markets missed it while experts got it. (And yes, I know there were others besides Silver and that is even worse for prediction markets)

        The fact the different predictions markets were making different predictions is also a huge strike against prediction markets. For markets to be efficient, you need a host of conditions to be met (which are typically not met in the real world). One of those is negligible transaction costs; and that can’t possibly apply when you have different predictions in different open markets.

      • http://www.gwern.net/ gwern

        > My big question is: are there

        *any* successes?

        Um, yes, there are: there have been hundreds of contracts on IEM, Intrade, and others showing excellent accuracy across all sorts of subject matters. That is what counts as success for something dealing with predictions. I’ve already linked to Scholar on prediction markets. If you aren’t impressed by the existing papers, I’m not sure what you could be impressed by or how you would expect anyone to find such evidence.

        > Likewise with Obama-McCain.

        Really? My understanding was that on election day – the day from which I was drawing predictions and getting that 2012 60/40 prediction – Intrade in 2008 (when US bettors had not been effectively banned from Intrade) was forecasting pretty high for Obama’s victory.

      • IMASBA

        “Really? My understanding was that on election day – the day from which I was drawing predictions and getting that 2012 60/40 prediction – Intrade in 2008 (when US bettors had not been effectively banned from Intrade) was forecasting pretty high for Obama’s victory.”

        Yeah, on election day. A problem with election prediction markets is that they only start becoming accurate close to the election itself. That said, I do believe that election prediction markets with enough volume are less worse than flipping coin.

      • http://www.gwern.net/ gwern

        > A problem with election prediction markets is that they only start becoming accurate close to the election itself.

        Citation needed. You used an example from one election day, I refuted it with another election day because that’s what you were basing it on. Now you are moving the goal posts to claim that actually prediction markets are accurate but only toward the election and they’re inaccurate all the other times? At least have the courtesy to try to justify your constantly shifting claims.

      • IMASBA

        Really? Nate Silver predicts more than a day ahead and I assume most people will find prediction markets useless if they only predict one day ahead, that’s not really a stretch or moving the goalposts.

      • http://www.gwern.net/ gwern

        Yeah it is, when that was what you were citing as evidence of inaccuracy. You’re moving the goalposts. So to repeat: where is your evidence that prediction markets are inaccurate before election day compared to pundits?

      • Curt Adams

        The scholar stuff is all theoretical. There’s no real-world data, except one saying prediction markets were better than some individual polls, which is not relevant. If you’ve got anything on prediction markets beating other systems on all sorts of subject matters, I’d be happy to hear it.

      • http://www.gwern.net/ gwern

        Wow, I’m impressed that you managed to read through all 4000 results and dismiss the Intrade, IEM, HSX, Google, Betfair etc results are being “no real-world data”. Well, I will just have to defer to you on this matter since you’re clearly far more familiar with the topic than me.

  • Tom Hynes

    Nice, very subtle dig at Obamacare. Your suggestion is that if we had a prediction market on “Successful October 1 Obamacare Launch” Obama would have realized a problem was coming up.

  • http://juridicalcoherence.blogspot.com/ Stephen Diamond

    Consider this argument. (I admit, a bit overstated.):

    If competitive investment markets haven’t driven companies to adopt this great innovation, then investment markets must be pretty inefficient. If investment markets are inefficient, there’s no reason to think prediction markets are accurate.

    • Jess Riedel

      Robin doesn’t argue that prediction markets are infallible, only that they are better than all the alternatives. Likewise, pointing out inefficiencies in corporations due to human biases isn’t an argument against capitalism.

      • IMASBA

        That’s not what Stephen was getting at. If corporations are so heavily influenced by human biases then why won’t prediction markets be? It’s jsut really weird that Robin wants the government to try out something first when the free market isn’t willing to while that something is based on the idea that the free market is vastly superior to the government. It’s very reminiscent of teabaggers saying the US doesn’t need social security because people are basically doing okay because of social security.

      • Jess Riedel

        >If corporations are so heavily influenced by human biases then why won’t prediction markets be

        Huh? Again, the claim isn’t that free markets are free from non-rational influences, the claim is that they are better than the alternatives.

        >It’s jsut really weird that Robin wants the government to try out something first when the free market isn’t willing to while that something is based on the idea that the free market is vastly superior to the government.

        “Free markets” are not the same thing as prediction markets, and your dramatic over-simplification to equate them doesn’t help anyone think clearly about this.

      • IMASBA

        “Huh? Again, the claim isn’t that free markets are free from non-rational influences, the claim is that they are better than the alternatives.”

        As the discoveries of biases in free markets continue to pile up and include significant stuff such as the free market not even figuring out they can use prediction markets then that claim becomes increasingly more difficult to defend. Also, it just creats a false sense of security. The least worst option is usually still pretty bad and not worth the effort.

        ” “Free markets” are not the same thing as prediction markets, and your dramatic over-simplification to equate them doesn’t help anyone think clearly about this.”

        The relevant properties proponents of futarchy cite are the same: a price mechanism, a eventual correction by reality and aggregation of opinions. The whole idea comes from looking at how financial markets predict things (where they forget about the quite different dynamics: a fantasy market predicting the housing prices in Buenos Aires, would yield different prices than the actual housing market in Buenos Aires, because the latter will directly influence the real housing prices).

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        Unfair! Robin uses the alleged efficiency of investment markets as evidence that prediction markets are good at aggregating information.

      • VV

        Huh? Again, the claim isn’t that free markets are free from
        non-rational influences, the claim is that they are better than the
        alternatives.

        Then the argument appears to have been essentially empirically falsified, given that Silver and Wang managed to do better than Intrade using only publicily available information and known statistical methods.

      • http://www.gwern.net/ gwern

        As already pointed out, that bad performance was predictable from the limits Intrade was operating under! And you only get that much by cherrypicking them out of the full set of predictors.

      • VV

        What cherry picking? Were there many other statisticians with a track record similar to that of Silver and Wang who tried to predict the election and performed worse than Intrade?

        And what limits was Intrade operating under? Regulation made betting difficult for Americans, but all the information used by Silver and Wang was publicly available to non-Americans, so why did they do worse than Silver and Wang?

        You could claim that Intrade was already small to begin with, and the near-exclusion of Americans made it too small to be efficient, but that invites the question of why Intrade wasn’t any bigger.

      • http://www.gwern.net/ gwern

        > What cherry picking? Were there many other statisticians with a track
        record similar to that of Silver and Wang who tried to predict the
        election and performed worse than Intrade?

        Even in 2012, Intrade was not the worst normal predictor (Margin of Error did worse with some regular models), and as I’ve pointed out ad nauseam now, the Intrade 2012 results were predictable in advance to be inferior solely on the fact that American bettors – like myself – could not trade on the markets.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        the claim isn’t that free markets are free from non-rational influences, the claim is that they are better than the alternatives.

        Then the appropriate experiment must pit prediction markets against alternatives. But that isn’t the rationale proponents present here; it seems enough that prediction markets are “accurate.”

        A real test, for example, would pit a prediction market against pollsters. We’ve seen a long argument about whether the top pollsters did better or worse than the prediction markets, but—it amazes me that the point is apparently unrecognized—this is no unbiased contest: the investors/gamblers were free to use the results of the polls!

        Can you imagine a prediction market for electoral results where there are no polls? (I mean, as a thought experiment.) Isn’t it fairly obvious that prediction markets without polls would perform worse than polls without prediction markets?

        Then why does it make much sense to recommend to industry that they use prediction markets instead of experts for figuring out how to aggregate information? (I think one reason is that experts can be yes-men. But that problem [to the extent anyone is serious about solving it] might better addressed directly by institutional protection for expert opinion.)

      • IMASBA

        “This is no unbiased contest: the investors/gamblers were free to use the results of the polls!”

        Yes, and that explains why election prediction markets only become accurate near, or even on, election day. On the other hand, why should it matter what sources prediction markets use if you just want information about the future (at least as long as no one is stupid enough to reason that since prediction markets work polls can be eliminated)?

      • http://www.gwern.net/ gwern

        > Yes, and that explains why election prediction markets only become accurate near, or even on, election day

        Cite or GTFO.

      • IMASBA
      • http://www.gwern.net/ gwern

        Criticism based on a single pair of contracts on a single day where it’s not obvious the movements were even wrong. This doesn’t even begin to rise to level of a statistical demonstration that prediction markets are inefficient up until Election Day – which was your claim!

        Epic fail.

      • oldoddjobs

        Oh Christ just fucking disagree without the fucking petulance oh there I go again

        LOL U EPIC FAIL NO CITE

        WHERE’S YOUR STUDIES, BRAH?

        U FOOL

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        It matters if you want to know whether industry and government should invest resources in prediction markets or in predictive expertise (represented, in this “experiment” by pollsters). (The electoral predictions are the most visible arena where prediction markets are said to have succeeded.)

        Robin says a prediction market would have given Obama advance notice of his problems with the healthcare site. Perhaps, but would not an administration less hostile to whistleblowers have gotten better technical information to the President? (In other words, expert opinion [analogous to pollsters] is preferable to the output from prediction markets.)

        In the end, it depends on their relative expensiveness. But the proponents of prediction markets seem to deploy a double standard. When faced by attacks on the efficiency of markets, they reply by saying that relative efficiency is the relevant consideration. But when it comes to their own evidence for prediction markets, they are uninterested in tests against the alternative approaches to prediction.

        Even if prediction markets would represent an improvement over today’s corporate decisionmaking, their achievements owe to overcoming the yes-man factor. And if industry were interested in overcoming that, there would (seemingly) be more efficient ways than using prediction markets. At least we don’t see the proponents here weighing prediction markets against less radical counters to the yes-man problem.

      • IMASBA

        “It matters if you want to know whether industry and government should invest resources in prediction markets or in predictive expertise (represented, in this “experiment” by pollsters).”

        You mean if there are only enough resources to conduct polls, or have a prediction market (or have a half-ass poll and a half-ass prediction market)? In that case, yes, you would definitely want to know what prediction markets can do without using poll data. I can see how such resource limitations could actually be a problem for certain other topics. It would definitely be useful to answer the question of spending the resources you would spend on an Obamacare website prediction market instead of spending that money on a commission of independent experts would yield better predictions or not. After all the prediction market does need some inside information, otherwise they predict the discovery of Saddam’s nukes again.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        Inasmuch as the alleged efficiency of markets is an argument for capitalism, their inefficiencies are arguments against capitalism.

        That may be merely semantic about “argument,” but surely it is an embarrassment to prediction markets (in evidentiary terms, not just a p.r. embarrasment) that their prototype—investment markets—have failed to drive their adoption.

        You should be willing to admit that much.

  • chepin

    There is typo in the title Furarchy > Futarchy

    • http://overcomingbias.com RobinHanson

      Fixed; thanks.

  • http://priorprobability.com/ prior probability

    Wouldn’t an internal (within-firm) prediction market be too thin to work? Also, without anonymous trading, what incentive would employees have to reveal information by trading on such markets?

    • http://overcomingbias.com RobinHanson

      There have been many dozens of internal prediction markets that have worked fine. With a market maker such markets can work fine with only a handful of traders. Even when individual trades are anonymous, the overall trader score need not be anonymous.

  • Pingback: Overcoming Bias : Fail Faster