Prediction Markets “Fail” To Mooch

What is new about prediction markets? To many, the key new idea is “crowd-sourcing”, which to many means that if you can “gamify” your problem enough, hobbyists will solve it for fun, much cheaper than if you had to hire employees. To me the key new idea is instead the “information prize”, a way to offer to pay others to find the info you want. (If they don’t find, you don’t pay.)

Intrade is a typical gamified market – you might pay Intrade a little to put up a question, and if traders like it they’ll pay Intrade to answer your question. A market-maker-driven corporate prediction market (such as firms pay Consensus Point to set up) is very different – to answer key business questions, firms pay lots to create markets, to fund employee participants, and to subsidize market makers.

Alas there’s been a lot more interest over the last few years in the get-work-for-free concept than in the pay-for-info concept. And more alas, recent discussions of “prediction market failures” are mostly on their failures to mooch. Case in point #1, Casey Mulligan today:

The efficient-prediction perspective presumes that the market exists on a scale large enough to create significant rewards for those with accurate predictions. Another perspective, “no-trade,” says prediction market participation will be low, if not zero, because traders suspect that a person would take the opposite side of their trades only because he had superior knowledge about the outcome. A market cannot survive if its only participants are “insider traders.”

Paradoxically, a prediction market cannot succeed unless it multitasks – it must serve an additional purpose separate from predictions, so that the participants with information about the outcome have counterparts to take the other side of their trades. In the case of sports and political markets, that additional purpose is entertainment – people enjoy engaging in the activity and are willing to participate even if their expected profits are zero or negative. These people are participating for various reasons other than prediction. (more)

No, a market can single-task, with no other function than prediction, and no other trader motive than selfish financial profit, if someone who wants the info will pay to subsidize the market. It is only when you want people to answer your question for free that you’ll have to piggyback on their having some other reason to trade.

Of course there’s no guarantee that your willingness to pay for some info exceeds other folks’ cost to supply that info. Supply and demand curves need not intersect at a positive quantity. But that’s hardly a failure of an info exchange mechanism.

Case in point #2, Snowberg, Wolfers, & Zitzewitz’s new paper “Prediction Markets for Economic Forecasting“, for the Handbook of Economic Forecasting, also sees not getting stuff for free as “failure”:

3.1 Why They (Sometimes) Fail

Although prediction markets generally function quite well, design flaws sometimes prevent reliable forecasts. These flaws generally lead to a lack of noise traders (or thin markets) that reduces incentives for discovering, and trading on the basis of, private information. In order to attract noise traders, the subject of a prediction market must be interesting and information must be widely dispersed. Prediction market contracts must be well specifed, so that it is clear when they will (and will not) pay off. However, this specicity may be in tension with making a contract interesting for traders. …

Noise traders may quite rationally choose not to trade in markets where there is a high degree of insider information. For example, despite the high intrinsic interest in who a Supreme Court nominee will be, markets on this topic have routinely failed. This may be due to the fact that most traders are aware that there are very few people with actual information on who the President’s choice will be. This anecdote underlines the importance of prohibiting insider trading: for instance, a market to predict the Institute for Supply Management’s (ISM’s) business confidence measure would be unlikely to function if it were well known that ISM employees were trading in it. …

Corporations are attracted to prediction markets as they can potentially pass unbiased information from a company’s front-line employees to senior management. However, many questions of interest to executives are not widely interesting, nor are there many employees that have relevant information. This creates a lack of liquidity in markets, perhaps leading to no trading, or, worse, inaccurate predictions. Microsoft has responded to this problem by using a market-making algorithm. ….

The stories of failure above lead to some straight-forward rules for designing prediction markets: make sure the question is well-defined, that there is dispersed information about the question, and that there is sufficient interest in the question to ensure liquidity. (more)

Noise traders are traders who subsidize your market for free, for reasons of their own, such as risk-hedging, idiocy, etc. If you fail to attract noise traders, you fail to get their free subsidy. But you can still offer to directly pay for your info, by subsidizing the market, as the Microsoft sentence in the quote indicates. Similarly, if employees find executive questions uninteresting, that just means they won’t answer such questions as freely in their spare time. But that hardly means firms can’t pay employees to address key firm questions. Here we are only talking about a “failure” of prediction markets to mooch stuff for free!

Snowberg, Wolfers, & Zitzewitz also err in saying that sometimes there is “no” info:

An extreme form of information not being widely dispersed is when there is no information at all to aggregate. For example, prediction markets on whether weapons of mass destruction (WMDs) would be found in Iraq predicted they would very likely be found. The false confidence that could be inspired by such an estimate ignores the fact that there was no information being aggregated by these markets. That is to say, it was unlikely that anyone in Iraq, who might actually have some information (perhaps based on rumors, past experience, or informal discussions with friends and relatives in the government) about whether Iraq’s WMD program was likely to exist or not, was trading in these markets.

Yes particular info, such as direct personal observations of WMD efforts, existed out there somewhere, but was prohibitively expensive to supply. That is, the price to buy info offered by the Intrade markets was too low to induce folks with such info to supply it. But that hardly meant there was no info available on the subject! There were a great many cheap but relevant clues available for making rough guesses on the subject, and I’m pretty sure that a lot of trading in that market was based on such clues.

When you offer to pay a certain price for info, an efficient info exchange mechanism will typically induce some supply of that info, but only up to the point where the marginal cost of supplying info reaches the price you have offered to pay. It is no failure of an exchange mechanism when buyers cannot always buy everything they want at as low a price as they want.

Finally Snowberg, Wolfers, & Zitzewitz end with this stunner:

We believe the real promise of prediction markets comes not from their ability to predict particular events. Rather, the real promise lies in using these markets, often several at a time, to test particular economic models, and use these models to improve economic forecasts.

This seems like saying the real promise of democracy is that academics can study votes to refine their theories of human behavior. Really?!

Added: Alas The Economist blog swallows this “no info” error whole.

Added 20July: Scott Sumner says people similarly say that his NGDP futures proposal “fails” if they don’t come for free.

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • Dean Jens

    Do you think democracy has some other value?

    • Michael Vassar

      You beat me to it.

      That said, yes, it’s cheaper and easier to win an
      election than to win an armed revolution, and the collateral damage is
      much lower, so assuming that you would end up with the same leaders in
      either scenario, Democracy would strictly dominate military
      dictatorship.  It doesn’t actually achieve that standard of strict
      domination, as we see in cases like Turkey, but it seems to me to be at least as plausible that in developed Western nations Democracy produces better leaders than military dictatorships do on average than that it does worse on average, causing the scale to favor Democracy even more strongly. 

      I really wish I saw more extremely half-hearted endorsements of the conventional wisdom.  Casually, I expect that these are what honest analysis would usually ultimately arrive at.

      • officer_fred

        > I really wish I saw more extremely half-hearted endorsements of the conventional wisdom.

        At first, I agreed with this, but then I realized that they’re everywhere. Show me a crowd of people, and I will show you as many walking, talking extremely half-hearted endorsements of the conventional wisdom as you could ever want.

  • http://entitledtoanopinion.wordpress.com TGGP

    I thought this from Miles Kimball/Noah Smith, on how the no-trade theorem doesn’t occur unless you explicitly remind traders of it, was interesting and of some relevance.

  • Pingback: TheMoneyIllusion » Robin Hanson on prediction markets

  • Wiki

    Isn’t there a problem here? How will we know a market on any specific prediction is subsidized enough to give efficient predictions in a neutral, ungamed fashion? Won’t badly setup trades leech off the success of “good” questions while being gamed? And how would an observer know that the markets set up by managers to test ideas are neutral? Wouldn’t the same incentives to rent seek also hold for companies setting up internal prediction markets? Especially if subsidies ate required?

    • http://overcomingbias.com RobinHanson

      There is no minimum subsidy required for efficiency or for “ungamed.” The attempts to game are themselves a subsidy and thus increase accuracy.  http://marginalrevolution.com/marginalrevolution/2008/10/manipulation-of.html

  • stevesailer

    The failure of prediction markets regarding Iraq’s nuclear nonprogram remains a major black eye. It’s not that there was no information — Here’s physicist Gregory Cochran’s 2002 correct analysis of the situation:

    http://www.jerrypournelle.com/archives2/archives2mail/mail227.html

    It’s that prediction markets analyzed the information wrongly. 

    Subsidy is hardly the solution, since there were large subsidies by pro-war interests to get the answer wrong.

    • http://overcomingbias.com RobinHanson

      There were subsidies to get the wrong answer in other forums, but not in the prediction markets. In prediction markets, the existence of manipulators willing to lose money to create a mistaken impression *is* a subsidy to other traders.

  • Philo

    Wiki already made my point:  The WMD case reminds us that the answer generated by a
    prediction market may depend on the amount of the subsidy.  If the people with the best information also
    have a very high cost of supplying that information, the subsidy required to
    get a good prediction will be very high; a market with a lower subsidy, though it seems to be functioning properly, will actually produce poor predictions.  And it may be difficult to determine how
    large a subsidy is needed to produce predictions of the desired quality.