Tag Archives: Prediction Markets

Better Bill Scoring

By Robin Hanson · September 19, 2009 12:15 pm · Comments (29)

Megan McArdle says med reform will pass, via bills exploiting CBO budget scoring errors:

I now put the chances of a substantial health care bill passing at 75%, and the chances of the Democrats losing the house in 2010 at about 66%. … The real game changer is that the CBO is willing to score health care savings on the grounds that the bill contains automatic spending cuts.

Conservatives are filled with rage and anguish. …  They are absolutely right:  the savings cuts will not be made, and I doubt that many in the Democratic party leadership, or the liberal wonkosphere believe that they will. …  The fact that the CBO has minimal discretion and uses roughly the same standards for every analysis is, despite its problems, a feature rather than a bug.  We may not like the fact that the CBO scores what’s in the law, rather than what is most likely to happen.  But the alternative is what?  An agency that can give the thumbs up or thumbs down according to how it feels about the legislators? …

This will make it very hard to keep the bill from passing, because legislators are, natch, more concerned about the appearance of fiscal rectitude than actual conservative budgeting.  … The public is probably going to accept the CBO numbers.

The alternative is prediction markets.  Compared to the value of making good decisions on these bills, or to the effort spent in “rage and anguish” on them, the cost to create prediction markets giving quality unbiased estimates of actual bill budgets would be small.

So why don’t now-loudly-wailing conservatives direct some of their energy to creating and promoting bill-scoring prediction markets?  Because they expect better bill budget estimates to make their positions look worse as often as better.  Sure, better estimates would help conservatives in this particular case, but they aren’t fool enough to think liberals lie about budgets more often than conservatives.

But aren’t there substantial organized political groups dedicated to uncovering and promoting the best policies, no matter whose ox they gore?  Apparently not.

Tagged as: ,

Words Vs. Bets

By Robin Hanson · September 11, 2009 3:00 pm · Comments (35)

I once had a long discussion with Ken Steiglitz about P=NP, while I was still at Princeton. … Ken was and still is sure that P must not be equal to NP. Okay, I said to Ken, what are the odds that they are equal? Ken said that he thought the odds were a million to one. I immediately suggested a bet. I did not ask him to “bet his life,” but I did ask for a million to one bet. I would put up one dollar. If in say ten years P=NP had not been proved, then he would win my dollar. If P=NP was proved in that time frame, then I would win a million dollars from Ken. Ken said no way. After more discussion the best bet I could get out of Ken was {2} to {1}.  Two to one. That was the best he would do.

That is from Richard Lipton; hat tip to Michael Nielsen.  Does anyone doubt that two to one better summarizes his evidence than a million to one?

Tagged as: ,

Prediction Markets As Collective Intelligence

By Robin Hanson · September 4, 2009 10:00 pm · Comments (5)

I talked for seven minutes this Wednesday at “Tap The Collective“, after six other speakers also talked for seven minutes each on various forms of “collective intelligence.”  I tried to put prediction markets (and similar mechanisms) in the context of other approaches by saying that other approaches often work very well when either:

  1. The info people contribute is verifiable, or
  2. The conclusions people draw are uncontroversial.

In these cases good tools, representations, interfaces, etc. can greatly help people join together in a spirit of constructive camaraderie to build documents, analyses, plans, etc.   People then appreciate the additions and edits of others in building a common product that all will admire.  False or misleading contributions can be quickly detected and eliminated.

The big problems for most collective intelligence tools come when the topics are controversial, and the contributions involve a lot of judgment.  For example, consider folks elaborating a schedule of which projects will be finished when, or designing a budget of which potential projects shall be funded.  Here folks are often justly concerned that many “contributions” will be self-serving attempts to make them or their groups look better or gain more resources.

Prediction markets were designed exactly these sort of hard problems – contributors know they face a risk of losing as well as gaining from their contributions.  So folks think a little more carefully about what they might say, and choose not to speak when they doubt they have something useful to say.  Prediction markets allow organizations to tap the collective to aggregate info on their most important and controversial topics.  But of course they aren’t the only or best way to support collaboration on all topics.

Cross-posted at Consensus Point.

Tagged as:

Tetlock Wisdom

By Robin Hanson · August 25, 2009 11:30 pm · Comments (6)

Both private- and public-sector prognosticators must master the same tightrope-walking act. They know they need to sound as though they are offering bold, fresh insights into the future not readily available off the street. And they know they cannot afford to be linked to flat-out mistakes. Accordingly, they have to appear to be going out on a limb without actually going out on one. That is why … they so uniformly appear to dislike affixing “artificially precise” subjective probability estimates to possible outcomes—the only reliable method we have of systematically tracking accuracy across pundits, methods, time and contexts. It is much safer to retreat into the vague language of possibilities and plausibilities—things that might or could happen if various difficult-to-determine preconditions were satisfied. The trick is to attach so many qualifiers to your vague predictions that you will be well positioned to explain pretty much whatever happens. China will fissure into regional fiefdoms, but only if the Chinese leadership fails to manage certain trade-offs deftly, and only if global economic growth stalls for a protracted period, and only if . . .

More here.  Hat tip to Henry at Crooked Timber.

Philip Tetlock seems to suggest that prognosticators are fooling us via this strategy, as if we would not tolerate such gaming if only we understood what they were up to.  I fear that instead we don’t much mind these games.  I suspect that we mostly want to affiliate with impressive folks, and reading their provocative forecasts gives us yet another excuse to do so.  As long no one else notices their failed forecasts enough to make them seem less impressive, we don’t really care if they were proved right or wrong.

Tagged as: ,

Futarchy in BBC Focus Mag

By Robin Hanson · August 10, 2009 1:30 pm · Comments (13)

I have a 600 word pro-futarchy oped in the current issue of BBC Focus magazine. It begins:

Has the recent MP expenses scandal soured the idea of democracy for you? Good, because a vast space of possible forms of government remains unexplored, and it is high time we explored it. Yes, democracy beats a dictatorship, but there might be better systems. …

Tagged as:

How Fix Boards?

By Robin Hanson · June 24, 2009 6:00 am · Comments (9)

In his book The Wisdom of Crowds, James Surowiecki held out hope that prediction markets could reform corporate information flows, and he was kind enough to mention me as an early innovator.  In his New Yorker column recently, he discussed fixing corporate boards:

Over the past couple of decades, a tremendous amount of attention has been devoted to improving corporate boards. New regulations, along with pressure from big investors, have forced companies to appoint more independent directors—people who have no direct connection to the company—and have tightened the definition of independence.  … All these changes, though, have had a much smaller impact than expected. …

There are ways to make boards proactive and more than nominally independent. Investors need to be able to play a much bigger role in determining who ends up on boards, nominating candidates themselves, instead of choosing among the C.E.O.’s picks. … On top of that, … big institutional investors [should] create a cadre of full-time directors, people whose only job would be to sit on corporate boards and look after shareholder value.

Problem is, investors need to own non-trivial fractions of companies to make this worth their while, and so this proposal needs a lot more concentration among investors than we have now, and probably more than most folks are comfortable with.   In contrast, my proposal to use prediction markets to advise key board decisions like firing a CEO requires no investor concentration.

I don’t really know if Surowiecki likes my proposal, or even knows of it.  He’s never returned my emails, though maybe he’ll see this post.  I suspect that he sees my proposal is too “out there” to befit a respected New Yorker columnist, and so wouldn’t endorse it even if he knew of and privately liked it.

Tagged as: ,

Reinventing Idea Futures

By Robin Hanson · May 27, 2009 6:00 am · Comments (33)

From the April Physics World:

A key problem, suggests mathematical physicist Eric Weinstein of the Natron Group, a hedge fund in New York, is that it is too easy for scientists in the “establishment” of any field to cut down new ideas, and to do so without really putting anything at risk, thereby leading to a culture that is systematically biased toward caution. …

Weinstein suggests another idea — that we should borrow some ideas from financial engineering and make scientists back up their criticisms by taking real financial risks. You think that some new theory is utterly worthless and deserving of ridicule? In the world Weinstein envisions, you could not trash the research in an anonymous review, but would buy some sort of option giving you a financial stake in its scientific future, an instrument that would pay off if, as you expect, the work slides noiselessly into obscurity. The money would come from the theory’s proponents, who would similarly benefit if it pans out into the next big thing.

Weinstein’s point is that markets, in theory at least, work efficiently and — putting the current financial meltdown to one side — lead to the accurate valuation of products. They exploit the “wisdom of crowds”, as a popular book of the same title recently put it. Take the famous electronic prediction markets at the University of Iowa, which pool the views of thousands of diverse individuals and consistently seem to give better predictions than any expert. …

“It would be more efficient,” he says, “if the maverick could demand of the critic, if my theory is so obviously wrong, why don’t you quantify that by writing me an options contract based on future citations in the top 20 leading journals secured by your home, furniture, holiday home and pension?”

This article makes it seem like Eric reinvented idea futures.  Except that Eric and I discussed the concept last May, when we had two phone conversations and exchanged seven emails.

In 1996, a Russ Ray published a paper in Futures Research Quarterly that was basically cut and paste from my Idea Futures paper.  Imitation is the sincerest form of flattery, right?  Hat tip to Jef Allbright.

Tagged as: , ,

Reply to Moldbug

By Robin Hanson · May 22, 2009 6:00 am · Comments (73)

A Mencius Moldbug has written a confused and rambling 7400 word critique of futarchy. But since Mencius seems to have passion and potential, let me try to communicate.  Most readers may prefer to skip this post; it will get tedious.

Prediction markets are a fine idea, whereas decision markets are… well… retarded. … Almost every conceivable application of a decision market … does not produce accurate predictions.

So if PM good, DM bad, your complaints should focus on features that distinguish decision and prediction markets, right?

For a market to produce accurate predictions, there must be genuine experts in the market, and they must be substantially better-funded than the morons.

OK, except that morons may largely cancel each other, in which case you don't need as many non-morons.  But this issue applies equally to prediction and decision markets, doesn't it?

If no one has ever seen the Emperor of China's nose, can a prediction market predict its length? … The worst case is that in which nobody has any way of actually calculating the prediction, but no one in the market is sure that this is the case. Your market signal will look exactly like that of an accurate prediction market, but predict nothing at all.

You can ask for a full probability distribution.  If speculators know they don't know anything, then they will give you a broad distribution that expresses a lot of uncertainty.  This is them telling you they don't know much.  In your nose example, they may just give you the distribution over nose sizes for elite Chinese.  And how is this issue different for prediction vs. decision markets?

Continue Reading "Reply to Moldbug" »

Tagged as:

Meta Institutions

By Robin Hanson · April 26, 2009 11:30 am · Comments (8)

Institutions are stable social contexts which make and coordinate actions.  Examples include elections, agencies, courts, clubs, debates, peer review, malls, games, media, etc.  It is by now an economic truism that institutions matter a lot.  Good institutions can induce good choices and info sharing, while bad institutions do the opposite.  

Rather than advise particular choices, economists prefer to advise on general policies, that apply to many choices.  We prefer even more to advise on choice of institutions, since an institutional choice can influence a great many policies.  Following this meta line of reasoning, we should prefer even more to advise on meta-institutions, i.e., institutions that structure our choices of institutions. 

We allow most of our familiar institutions to at least influence our institutional choices.  But no doubt we use some more often in that role, and some are better suited to that role.  While I'm excited that decision markets can help advise organization decisions, I'm most excited about their potential as meta-institutions, advising us on key policy and institutional choices.  Of course we'll have to demonstrate their effectiveness more on small issues before folks will rely on them for big issues.

Some basic questions:

  1. What institutions are especially good as meta-institutions?
  2. What institutions should we use to evaluate meta-institutions?
  3. What institutions are biased to prefer other institutions like themselves?
  4. How often do different institutions agree on particular institutional choices?
  5. What institutions can sensibly say if to rely on them as meta-institutions?
Tagged as: , ,

Wisdom of Rhode

By Robin Hanson · April 24, 2009 6:00 am · Comments (3)

Paul Rhode's "skeptical perspective" on corporate prediction markets:

In many ways, tapping of the wisdom of crowds within the firm is intended to overcome the information barriers created by the bureaucracy.  It is obvious that the upper management might want better access to selected information available down the organizational ladder, to stop having to listen to the self-serving lies of middle management.   But it seems to me, the individuals in an organization derive their power from the information under their exclusive control and will not easily give up this monopoly position.  What models we economists have about hierarchies largely concern controlling information flows, both up and down the organization.  This includes both having the higher-ups monopolize the firms’ secrets and strategies and preventing them from being overwhelmed by the day-to-day minutia.

With internal prediction markets, key questions include who will set the agenda, who decides what questions will be answered and how? It seems authority matters in whether this is done in a top-down or bottom-up manner. If the question is what is the best forecast for demand growth, will this deadline be met, or how will the product rank in quality tests, it is clear that upper management, the “deciders,” would be happy to learn from the collective wisdom of employees in contact with customers or doing the design work.  If the questions posed address how long before the company president is fired, whether this product is found defective and has to be recalled, or when the mass layoffs will begin, then upper management will be unhappy.

Prediction markets provide more information, but they do so in a public way.  What prevents competitors from spying, from gaining access to company secrets?  Besides making private information common knowledge, prediction markets undermined the mystique, the information monopoly of those in charge.

I agree completely and have said similar things many times.  So why am I called a "hyperbolic" optimist?  Today I speak at a corporate prediction markets summit in NYC.

Tagged as: