Overcoming Bias

Lobbying for Prediction Markets

Today I first appear in Science as an author (last time I was news), though I have eighteen coauthors.  It is a Prediction Markets "Policy Forum" (ungated here): 

Prediction market prices can be used to increase the accuracy of poll-based forecasts of election outcomes, official corporate experts' forecasts of printer sales, and statistical weather forecasts used by the National Weather Service.

Several researchers emphasize the potential of prediction markets to improve decisions. The range of applications is virtually limitless - from helping businesses make better investment decisions to helping governments make better fiscal and monetary policy decisions.

Prediction markets have been used by decision-makers in the U.S. Department of Defense, the health care industry, and multibillion-dollar corporations such as Eli Lilly, General Electric, Google, France Telecom, Hewlett-Packard, IBM, Intel, Microsoft, Siemens, and Yahoo. The prices in these markets reflect employees' expectations about the likelihood of a homeland security threat, the nationwide extent of a flu outbreak, the success of a new drug treatment, the sales revenue from an existing product, the timing of a new product launch, and the quality of a recently introduced software program.

These markets could assist private firms and public institutions in managing economic risks, such as declines in consumer demand, and social risks, such as flu outbreaks and environmental disasters, more efficiently.

Unfortunately, however, current federal and state laws limiting gambling create significant barriers to the establishment of vibrant, liquid prediction markets in the United States. We believe that regulators should lower these barriers by creating a legal safe harbor for specified types of small stakes markets, stimulating innovation in both their design and their use.

Coincidentally, on May 1 the CFTC asked for public commentary on regulating prediction markets - comments due July 7.

Should-not-be-needed disclaimer:  Such an endorsement says only that I estimate this change to improve on the status quo. 

If I Had A Million

I've been telling folks for a long time that if I had a million dollars to spend on a project, I'd implement an idea I posted first twelve years ago today, and described in Forbes sixteen months ago:

Make two subsidized real-money markets on the stock price of each Fortune 500 firm, one market conditional on its CEO stepping down by quarter's end, and the other conditional on not stepping down.  The difference between these two prices would advice the board on dumping the CEO.

If active, these markets should attract business press, and then most of these CEOs would come to see what the markets say about them.  Half a million would pay for legal/admin.  The other half would only cover a $1000 subsidy per firm, but CEOs trying to manipulate would add lots of liquidity.  A few years of data would let us clearly compare the returns of firms following market advice to firms not following.   With clear data I'd encourage shareholders to sue boards ignoring market advice, and after a few wins most boards would weigh market advice heavily.   A revolution in CEO accountability would then be complete, all for only a million.

Technically, I'd also create a third market per firm in the chance the CEO will step down, and force the three prices to be consistent with external stock prices, so there'd only be two independent degrees of freedom.

Added: And of course this would be a huge foot-in-door legitimization and exposure for many other kinds of organizational decision markets. 

Cash Increases Accuracy

From the current New Scientist:

In an experiment dubbed "Cola Wars", [Nick Epley] conducted a taste test with a twist: he told participants which cola was Coke and which was Pepsi before tasting began. After tasting, all they had to do was estimate what percentage of their friends would be able to distinguish between the two in a blind taste test. Studies show that people's ability to do this is no better than chance - so an answer around 50 per cent would be right. What Epley found was intriguing. When he motivated volunteers to give a considered response - by offering them a cash payment - their answers tended to be close to 50 per cent. Subjects who were not paid, however, seemed to answer with an egocentric bias: since they knew which cola was which, they assumed that a high proportion of their friends would guess correctly (Journal of Personality and Social Psychology, vol 87, p 327).  For Epley, the finding supports his idea that putting yourself inside the head of another person and considering their perspective requires a cognitive effort that simple egocentric judgements do not.

Make no mistake: stronger incentives often (though not always) make us see more clearly.

Colorful Character Again

I just learned of a new Scientific American article on prediction markets, which is pretty positive: 

A paper ... compares the performance of the IEM as a predictor of presidential elections from 1988 to 2004 with 964 polls over that same period and shows that the market was closer to the outcome of an election 74 percent of the time. ... Attracted by the markets' apparent soothsaying powers, companies such as Hewlett-Packard (HP), Google and Microsoft have established internal markets that allow employees to trade on the prospect of meeting a quarterly sales goal or a deadline for release of a new software product. As in other types of prediction markets, traders frequently seem to do better than the internal forecasts do. ... Prediction markets may truly hark back to the future. "My long-run prediction is that newspapers in 2020 will look like newspapers in 1920," Wharton School's Wolfers says. If that happens, the wisdom of crowds will have arrived at a juncture that truly rivals the musings of the most seasoned pundits.

But I am personally singled out as the colorful character who is way too positive: 

Talking Versus Trading

The latest Journal of Prediction Markets describes lab experiments comparing prediction markets to deliberation.  They collected 24 groups of four people to forecast a survey on the popularity of five cell phone designs.  Half the groups talked among themselves and then made a group forecast, with prizes going to the best groups.  In the other half of the groups the individuals rotated trading with an automated market maker, with prizes going to the best individual forecasters.  They found:

The direct comparison of the two group mechanisms shows that the variation across the twelve estimates is much lower for the Traders with an average standard deviation of 3.2 percent compared to 7.9 percent for the Talkers. 

I expect talking groups to do even worse when members have differing interests to manipulate the group conclusion.  Since prediction market accuracy seems to not be hindered by such manipulation, the difference between the two mechanisms should be even larger in that case.   

Buy Now Or Forever Hold Your Peace

The Intrade prediction market is giving Hillary a 53% chance and Obama a 47% chance of winning the Democratic presidential nomination.  Hillary is down 7.5 percentage points in just the last day.  (Note:  Between when I wrote the above, and when I posted this, Hillary went up to 54.)

From what I've read on Intrade, you can fund your account with up to $250 using a credit card, and it should land in your account immediately.  (More than this takes time.)  Also, remember that you can sell contracts at any time afterward - you don't have to wait months to collect your payout.

If you think that Hillary is going to do better than the polls on Super Tuesday, and you're going to sneer afterward and say that Intrade was "just tracking the polls", buy Hillary now.

If you think that Obama is going to do better than the polls on Super Tuesday, and you're going to gloat about how prediction markets didn't call this surprise in advance, buy Obama now.

If you don't do either, then clearly you do not really believe that you know anything the prediction markets don't.  (Or you don't understand expected utility, or your utilities over final outcomes drop off improbably fast in the vicinity of your current wealth minus fifty bucks - you don't have to bet the full $250.)  It is free money, going now for anyone who genuinely thinks they know better than the prediction markets what will happen next.

Prediction markets do not have supernatural insight.  If they give the candidates fifty-fifty odds, it means that the market collectively doesn't know what will happen next.  Even if you're well-calibrated, you get surprised on 90% probabilities one time out of ten.

The point is not that prediction markets are a good predictor but that they are the best predictor.  If you think you can do better, why ain'cha rich?  Any person, group, or method that does better can pump money out of the prediction markets.

If prediction markets react to polls, they're getting new information, that they didn't predict in advance, which happens.  Being the best predictor doesn't make you omniscient.

Everyone's going to find it real easy to make a better prediction afterward, but if you think you can call it in advance, there's FREE MONEY GOING NOW.

Buy now, or forever hold your peace.

Predictocracy vs. Futarchy

A last word in my debate with Michael Abramowicz.  He presents the choice between predictocracy and futarchy as a choice between errors in choosing values and errors in estimating consequences:

Hanson makes some strong points in favor of futarchy. "Democracy today suffers from enormous errors regarding estimates of policy consequences." ... Ex post evaluators in predictocracy might make some systematic errors that prediction market traders in futarchy would fix.  Futarchy, however, introduces another type of error, the danger that the legislature will not do a good job of defining GDP+, as Hanson acknowledges. It's not a priori clear which would be worse -- errors by the legislature in developing a formula for GDP+, or errors by ex post evaluators in determining whether a particular policy has increased or decreased general welfare.

But it is far from obvious that being forced to explicitly describe our values increases value errors in our decisions.  Our case specific intuitive judgments made without reference to explicitly described values also embody many value errors.  As I said in my last post:

Merger Decision Markets

HP began to explore prediction markets in 1996, but did not even consider applying them to the 2002 HP-Compaq merger.  Similarly, Yahoo and Microsoft are two of the companies mentioned most often as being involved in prediction markets (along with their main competitor Google), but I'll bet none are considering the by-far-most-valuable markets they could create, on their just-announced proposed merger.  Decision markets could say whether this merger is good for shareholders, by estimating the combined stock price given a merger, and given no merger.  Similarly, decision markets could say whether this merger is good for these firms' customers, by estimating the price and/or quantity of web ads given a merger, and given no merger.  This might help convince regulators to approve the merger.   From the Post:

The deal also will face scrutiny from antitrust regulators, who will hear from consumer groups concerned that so much power -- and consumer information -- would be concentrated in one company.   However, some said that allowing the acquisition would enhance competition by creating a credible competitor to Google.

"You want to encourage competition with Google," said Ben Scott, policy director at Free Press, a public-interest firm. "But you don't want to bend over backward to do that and end up with a duopoly that ignores the anticompetitive implications of Microsoft integrating Yahoo products into Windows."

The Justice Department and House and Senate committees are likely to review the transaction, officials said. "We will need to scrutinize the deal carefully to insure that it will not cause any harm to the competitiveness of what has been a vibrant high-tech marketplace," said Herb Kohl (D-Wis.), chairman of the Senate Judiciary Committee's subcommittee on antitrust, competition policy and consumer rights.

The subcommittee, which recently examined Google's proposed takeover of DoubleClick, would also look at whether a Microsoft-Yahoo deal would violate the privacy rights of Internet users.

My main doubt here is whether ad price and quantity are good enough measures of the merger's social benefits - what other outcomes could such markets estimate, to speak more clearly?  And this is a very clear demonstration that these companies are just not serious about finding the highest value applications of prediction markets.

Added: More comments at the cross -posting at Midas Oracle.   

Futarchy vs. Predictocracy

In 2000 I proposed "futarchy", a government by prediction markets where we would "vote on values but bet on beliefs."  In his 2008 book, Michael Abramowicz summarizes:

At the heart of Hanson's proposal is the use of conditional markets to estimate the effect of proposed policies on a measure of national welfare. ... The existing legislature ... could pass legislation to change the welfare measure, producing what he calls GDP+ ... Any policy that, according to a prediction market, would clearly improve GDP+ would be automatically adopted.

In contrast, Abramowicz prefers "predictocracy": 

The essence of predictocracy is that decisions are made on the basis of the market's anticipation of whether they will ... be approved of later. ... Individuals in this legislature might be asked to vote not on values that others would then seek to achieve but instead on concrete statutory proposals, as existing legislatures do today. These legislators' decisions, however, would have no immediate effect but would serve only to discipline the text-authoring market, whose decisions would determine whether particular amendments were allowed and particular bills were enacted.

Why is this better?

Deliberative Prediction Markets -- A Reply

Robin suggests that a more robust model of deliberative prediction markets would be useful, and I agree. Experimentation in the field would be even more useful. But my original paper and book section explain the logic of this market approach clearly, with both words and what I acknowledged was a "simple model."

I doubt, in any event, that a more elaborate model will change the basic conclusion: that the deliberative prediction market provides at least some increased incentive to reveal information. I'll let interested readers look at the original paper for a more developed argument (including math), but it boils down to a very simple point. A prediction market, as Robin and I both note, already provides some incentives to reveal information. But if a trader's payoff depends on whether the trader actually succeeds at persuading others rather than on whether the trader turns out in the long run to be correct, the trader will have an additional incentive to reveal that information.