Tag Archives: Prediction Markets

I Talk Wed. At Harvard

By Robin Hanson · April 21, 2011 9:40 am · Comments (6)

Next Wednesday I’ll talk at Harvard business school on “Toward Information Accounting“:

What gets measured, gets done. Today, organizations account in great detail for revenue and the costs of materials and time, but have only crude informal accounting of info contributed to key organizational decisions. Because info cost and value are poorly measured, info production is neglected.

Can we use prediction markets to do better? Imagine speculative betting markets on many key organizational questions, and two key changes in business practice. First, let the division responsible for each decision declare lower-bound estimates of the value of more info on each related question. A division might, for example, declare that 1% lower error in estimating 3rd quarter sales of product X is worth at least $5000. There are standard ways to calculate such info value in specialized situations, such as inventory management.

Second, let trader accounts be denominated in a new “color of money.” Instead of doing zerosum betting, the market for each question would be subsidized at a level matching its declared info value. As a result, the subsidy amounts lost to traders as prices become more accurate would on average correspond to that question’s declared info value. For example, on 3rd quarter sales of product X, its 0.7% lower error might have earned a $3500 subsidy, going to George who gained $2000, Sue who gained $1500, Sam who gained $1000, and Fred who lost $1000.

Given these two new practices, trader account gains could be interpreted as noisy estimates of the info value those accounts transmitted via their trades. Losses could be interpreted as info destruction. Simple statistics applied to the pattern of changes in an account over time could estimate its consistent gains, amid its temporary fluctuations. The total consistent gains for the accounts of a division could be credited to that division in its ordinary cost accounting, while that same amount is debited from the divisions who declared info value on those questions.

When one created an account with an initial cash deposit, and authorized an individual or team to trade that account on specific questions, one would in essence say: “Try to show us that you can consistently add info value here via your trades. We’ve started you out small, but if you can show consistent gains we may give you more to work with. At annual review time we’ll credit your account’s consistent gains (or losses) to you (and your division) as value you transmitted to this organization, to be compared with your time and other costs of participation.”

GD Star Rating
loading...
Tagged as: , ,

What Do I Want To Know?

By Robin Hanson · April 18, 2011 8:00 am · Comments (18)

Reading the novel Lolita while listening to Winston’s Summer, thinking a fond friend’s companionship, and sitting next to my son, all on a plane traveling home, I realized how vulnerable I am to needing such things. I’d like to think that while I enjoy such things, I could take them or leave them. But that’s probably not true. I like to think I’d give them all up if needed to face and speak important truths, but well, that seems unlikely too. If some opinion of mine seriously threatened to deprive me of key things, my subconscious would probably find a way to see the reasonableness of the other side.

So if my interests became strongly at stake, and those interests deviated from honesty, I’ll likely not be reliable in estimating truth. Yet as my interests fade to zero, I also suspect my opinions to be dominated by random weak influences, such as signaling pressures, that also have little to do with truth. My reliability seems contingent on my having atypically good incentives to get it right.

So on what topics do I have good incentives? Of course this is also a subject on which I may have poor incentives for accuracy. If things precious to me depended on my believing I had good incentives, well then I’d believe that, even if untrue. What to do?

It seems my safest place to stand for drawing inferences is on my most robust beliefs about good incentives. And for me, that place is prediction markets. Since prediction markets seem to give robustly good incentives on a rather wide range of topics, I should believe what they say, and think I’d have more reliable beliefs if we had more such markets. I might think we don’t need them much on certain safe topics, because we already have good reliable other ways to estimate such topics. But I just can’t trust such judgements that much – they might also be biased.

Of course I can’t know that I or we will be better off by having more truthful estimates on any particular topic. I might think that on certain topics we’d be better off not knowing. But I can’t trust that judgement greatly – it would be better to rely on prediction markets on this meta question, of what we’d be better off not to know.

Someday hopefully we’ll have many prediction markets, and maybe even futarchies, to guide humanity through the many shoals ahead, including on what we’d do better not to know. Of course we might be mistaken about what we value, and so ask futarchies about the wrong consequences, thus inducing mistakes about what we’d rather not know. So it is especially important to consider the values in which we have the most confidence.

You might argue that your best estimate is that we are in fact seriously mistaken on what we value, so mistaken that we would ask futarchies the wrong questions, and then such markets would mislead us on what we’d be better off not to know. You might instead recommend that we follow your suggestions about what we should know, and what to believe in the absence of the prediction markets you advise against. And well, you might be right. But really, what grounds do you have have for confidence in that set of judgements? Why should we trust your judgement on the good quality of the incentives for your intuitions?

GD Star Rating
loading...
Tagged as:

Modular Innovation

By Robin Hanson · March 9, 2011 6:30 pm · Comments (27)

Seth Roberts says economists neglect innovation:

How to avoid or recover from stagnation … is the central question of economic life, with no clear answer. Yet it is roundly ignored. In the Berkeley Public Library a few years ago, I picked up an introductory economics textbook for junior colleges, 700 pages long. It had one page – fact-free, poorly-written – about where new goods and services come from. This is typical of the introductory economics textbooks I’ve seen. It reflects the profession as a whole: I estimate about 1% of mainstream economic research is about innovation. It should be half the field.

He’s right; innovation is neglected, at least using a standard of what has impact or relevance. But academics don’t study topics because they are important; they study topics to gain prestige, by being certified as mastering impressive techniques. Sure, all else equal it can help to write about an important topic.  (At least if you avoid taking on a topic too big for your status – big grand overviews and contrarian jabs tend to be reserved for senior folk.) But academics usually aren’t rewarded enough for the added effort to figure out what topics are actually important. So they might as well just do what others say is important. And since it is hard to use standard impressive tools to study how to promote innovation, that topic gets neglected.

Enough excuses; here’s a positive contribution. It seems to me that one of the major factors limiting innovation is this: would be innovators must now combine two risky decisions:

  1. What innovative ideas or projects are ripe and promising to purse now?
  2. Who is best placed or skilled to attempt the realization of each idea?

People who pitch project ideas to venture capitalists often focus on convincing them of #1, idea quality, not realizing that if you convince them of that but not #2, your team quality, they will just steal your idea and give it to another better team. Usually they hear from several teams pitching pretty similar concepts, so they are judging mainly on team quality.

Knowing this, sophisticated innovators tend to neglect idea quality, and focus on team quality. Naive innovators address both issues, but being naive they don’t know enough about what other folks think about the quality of their ideas. The net result is too little aggregation of info about idea quality. Could we do better?

Prediction markets, to the rescue! Imagine prediction markets on which innovative ideas will succeed soon, possibly conditional on approach or team style. Such prediction markets could offer a valuable modularity to aid innovation. Some people could focus on idea quality, and profit from their insights by trading in markets on which ideas will succeed when. Other folks could focus on team quality, by creating high quality teams which pursue the ideas that prediction markets have endorsed. Such teams could hedge some of their idea risk in prediction markets, and that hedging would add market liquidity, enabling idea specialists to better profit from their insights.

GD Star Rating
loading...
Tagged as: , ,

Me On Stossel

By Robin Hanson · February 16, 2011 3:50 pm · Comments (3)

I should appear briefly tomorrow on the John Stossel show, showing on Fox Business Thursday evenings at 9,12 EST, talking about prediction markets.

Added 18Feb: Video is here.  My segment starts about minute 13.

GD Star Rating
loading...
Tagged as: ,

CMU Talk Tuesday

By Robin Hanson · February 14, 2011 10:15 pm · Comments (3)

Tomorrow at 3:30 I’ll speak on “The Potential of Prediction Markets” at the Carnegie Mellon computer science department. More here and here.

GD Star Rating
loading...
Tagged as: ,

The Big Failure

By Robin Hanson · February 14, 2011 12:20 pm · Comments (26)

Scott Sumner:

Most of the really important public policy issues are not even part of the ongoing debate in the press. Here are some examples:

1. The huge rise in occupational licensing.

2. The huge rise in people incarcerated in the war on drugs, and also the scandalous reluctance of doctors to prescribe adequate pain medication (also due to the war on drugs.)

3. The need for more legal immigration.

4. The need to replace taxes on capital with progressive consumption taxes.

5. Local zoning rules that prevent dense development.

6. Tax exemptions for mortgage interest and health insurance.

These 6 policy failures impose enormous damage on the country, far more than the issues typically discussed on the evening news. Why aren’t they discussed? I would argue that it is partly because the disagreements tend to break down on values, not ideology. Most idealistic intellectuals agree with me on all of these issues. They are not issues that divide the left and the right. It’s also true that most real world politicians agree on these issues. However their views are exactly the opposite of the views of intellectuals. Hence there is no “policy debate” in either the political or intellectual arenas, and hence no “fight” for the media to report.

Adam Ozimek:

The missing piece of this puzzle is that the intellectual agreement on these issues isn’t just the opposite of real world politician’s, but the opposite of the rest of the real world. At the average dinner table in this country, anyone advocating what Sumner might call the intellectual consensus on any of these issues would face a lot of disagreement, and would frequently be greeted by surprise that a reasonable person would ever dream of advocating for, say, for more immigration or less occupational licensing.

The key questions are, of course, why is it so hard to inform the public that intellectual elites disagree with them on such issues, and if being informed of this fact would be enough to change their minds.

If telling the public that elites disagree would be enough to change their minds, well then a public info campaign targeting this ignorance could yield huge rewards. Then we’d face the question of why no philanthropists care enough to fund such a campaign. Could it be that they also mainly care about taking ideological sides?

Talking to the public may not be enough, however, if the public just does not want to hear that elites disagree with them. It is hard to tell folks things they do not want to hear. It might also be that even if the public does hear it, they would not change their minds. In which case democracy just loses.

A variation on democracy, like futarchy, that relies more on expert judgement on what causes what, could do better. But to get from here to there, you’d have to convince the public to accept a form of governance that relies more on something other than on their personal opinions. Not impossible, but not easy either.

GD Star Rating
loading...
Tagged as: ,

Bad News: Kant & Bets

By Robin Hanson · January 7, 2011 11:30 am · Comments (36)

The famous philosopher Kant saw bets as encouraging thoughtfulness and discouraging self-deception:

The usual touchstone of whether what someone asserts is mere persuasion or at least a subjective conviction, i.e., firm belief, is betting. Often someone pronounces his propositions with such confident and inflexible defiance that he seems to have entirely laid aside all concern for error. A bet disconcerts him. Sometimes he reveals that he is persuaded enough for one ducat but not for ten. For he would happily bet one, but at ten he suddenly becomes aware of what he had not previously noticed, namely that it is quite possible that he has erred. (Critique of Pure Reason, A824/B852; more; HT Tyler)

If we were to see life out there in the universe, at or below our level of development, that would be bad news regarding our future.  It would suggest that more of the great filter that stands between dead matter and expanding civilization lies ahead of our place on that path. Similarly, it is bad news to hear that Kant had a high opinion of the accuracy advantages of bets.  Let me explain.

I hope for a future where betting markets are a commonly used mechanism to create official consensus beliefs, but I must explain the fact that they are not already often used this way.  What barriers have stood in their way? One barrier is widespread skepticism about bet accuracy. But hearing of Kant’s well-known position reduces my estimate of this barrier; many respected people have long respected bet accuracy. So I must therefore increase my estimate of the difficulty of other barriers.  Alas, since skepticism about accuracy seems one of the easiest barriers to overcome, via track records and lab experiments, I must increase my estimate of the overall difficulty of my goal.  I’ll keep trying though.

GD Star Rating
loading...
Tagged as: , ,

How To Doubt

By Robin Hanson · January 5, 2011 6:00 pm · Comments (15)

Doubting conventional wisdom seems pretty central to my style and central issues. So I took some time to ponder the general issue of doubt. Here’s what I came up with.

The relation between confident and doubtful states of belief is like that between the three corners of a triangle and the points within the area of the triangle. Even though there are far more area points than corner points, area points are more similar to each other, in a distance sense. In the same way, there are far more states of doubt than confidence, yet states of doubt are more similar to each other, in that they lead to similar decisions.

What can you do about serious skepticism, i.e., the possibility that you might be quite mistaken on a great many of your beliefs? For this, you might want to consider which of your beliefs are the most reliable, in order to try to lean more on those beliefs when fixing the rest of your beliefs. But note that this suggests there is no general answer to what to do about doubt – the answer must depend on what you think are actually your most reliable beliefs.

When seeking your most reliable beliefs, to help fix other beliefs, you might be tempted to just consider simple beliefs like “George is my friend” or “carrots are orange.” But you may do better to rely on your less often stated beliefs that it is quite unlikely for a great many of your “independently” generated beliefs to all be mistaken.

That is, our most potent beliefs for dealing with doubt are often our beliefs about the correlations between errors in other beliefs. This is because having low error correlations can imply that related averages and aggregates are very reliable. For example, if there is little correlation in the errors your eyes make under different conditions in judging brightness, then you need only see the same light source under many conditions to get a reliable estimate of its brightness.

Since beliefs about low error correlations can support such strong beliefs on aggregates, in practice doubt about one’s beliefs often focuses on doubts about the correlations in one’s belief errors. If we guess that a certain set of errors have low correlation, but worry that they might really have a high correlation, it is doubts about such hidden correlations that threaten to infect many other beliefs.

So then what do we actually worry about, when we worry that our belief errors might be correlated? It seems to me that we mainly worry about two sources of correlated error:

  • Hidden psychological tendencies: We worry that our mind are built in such a way as to give related errors on what appear to be unrelated topics. Our minds might, for example, be biased toward high estimates of our ability, for many kinds of unrelated abilities.
  • Hidden social coordination: We worry that our social groups coordinate so as to give related errors from what appear to be unrelated social sources. Sources that share a common ideology might, for example, make similar errors on diverse topics.

Most academic consideration of radical skepticism happens in philosophy. But the above analysis suggests that if you were serious about actually doubting, instead of just discussing doubt, you’d want to study psychology and social sciences, especially hidden psychological biases and social coordination. Getting a grip on these subjects might position you well to actually consider the possibility that you might in fact be quite mistaken about a great deal.

Of course once you do understand psych and socsci, there is no guarantee that such understanding enables you to, on your own, powerfully address your doubts.  If fact, you may end up agreeing with me that our best approach is for would-be doubters to coordinate to support new institutions that better reward correction of error, especially correlated error.  I refer of course to prediction markets.

In sum: States of doubt are diverse, yet lead to similar decisions, relative to states of confidence. To productively doubt, you’ll want to identify beliefs in which you have greater confidence. When your belief errors have low correlation, you can have quite high confidence in certain aggregate beliefs. So doubts about belief error correlations are central to real skepticism. Since most doubts about correlations seem to arise from concerns about hidden mental tendencies and social coordination, a serious doubter will give those topics the most attention. And an ambitious doubter might join me in supporting something like prediction markets.

GD Star Rating
loading...
Tagged as: , , ,

Gambling Save Science?

By Robin Hanson · December 12, 2010 12:30 am · Comments (19)

The latest New Yorker:

All sorts of well-established, multiply confirmed findings have started to look increasingly uncertain. … This phenomenon … is occurring across a wide range of fields, from psychology to ecology. … The most likely explanation for the decline is … regression to the mean. … Biologist Michael Jennions argues that the decline effect is largely a product of publication bias. Biologist Richard Palmer suspects that an equally significant issue is the selective reporting of results. … The disturbing implication … is that a lot of extraordinary scientific data is nothing but noise. (more)

Academics are trustees of one of our greatest resources – the accumulated abstract knowledge of our ancestors. Academics appear to spend most of their time trying to add to that knowledge, and such effort is mostly empirical – seeking new interesting data. Alas, for the purpose of intellectual progress, most of that effort is wasted. And one of the main wastes is academics being too gullible about their and allies’ findings, and too skeptical about rivals’ findings.

Academics can easily coordinate to be skeptical of the findings of non-academics and low-prestige academics. Beyond that, each academic has an incentive to be gullible about his own findings, and his colleagues, journals, institutions, etc. share in that incentive as they gain status by association with him. The main contrary incentive is a fear that others will at some point dislike a findings’ conclusions, methods, or conflicts with other findings.

Academics in an area can often coordinate to declare their conclusions reasonable, methods sound, and conflicts minimal. If they do this, the main anti-guillibility incentives are outsiders’ current or future complaints. And if an academic area is prestigious and unified enough, it can resist and retaliate against complaints from academics in other fields, the way medicine now easily resists complaints from economics. Conflicts with future evidence can be dismissed by saying they did their best using the standards of the time.

It is not clear that these problems hurt academics’ overall reputation, or that academics care much to coordinate to protect it. But if academics wanted to limit the gullibility of academics in other fields, their main tool would be simple clear social norms, like those now encouraging public written archives, randomized trials, controlled experiments, math-expressed theories, and statistically-significant estimates.

Such norms remain insufficient, as great inefficiency remains. How can we do better? The article above concludes by suggesting:

We like to pretend that our experiments define the truth for us. But … when the experiments are done, we still have to choose what to believe.

True, but of little use. The article’s only other suggestion:

Schooler says “Every researcher should have to spell out, in advance, how many subjects they’re going to use, and what exactly they’re testing, and what constitutes a sufficient level of proof.”

Alas this still allows much publication bias, and one just cannot anticipate all reasonable ways to learn from data before it is collected. Arnold Kling suggests:

An imperfect but workable fix would be to standardize on a lower significance level. I think that for most ordinary research, the significance level ought to be set at .001.

I agree this would reduce excess gullibility, though at the expense of increasing excess skepticism. My proposal naturally involves prediction markets:

When possible, a paper whose main contribution is “interesting” empirical estimates should give a description of a much better (i.e., larger later) study that, if funded, would offer more accurate estimates. There should be funding to cover a small (say 0.001) chance of actually doing that better study, and to subsidize a conditional betting markets on its results, open to a large referee community with access to the paper for a min period (say a week).  A paper should not gain prestigious publication mainly on the basis of “interesting” estimates if current market estimates of better estimates do not support those estimates.

Theory papers containing proofs might similarly offer bets on whether errors will be found in them, and might also offer conditional bets on if more interesting and general results could be proven, if sufficient resources were put to the task.

More quotes from that New Yorker article: Continue reading "Gambling Save Science?" »

GD Star Rating
loading...
Tagged as: ,

Compare Refuge, Resort

By Robin Hanson · December 10, 2010 2:00 pm · Comments (17)

Wednesday I gave a brief talk (audio, slides) at the annual meeting of the Society for Risk Analysis. It seems many risk analysts are like futurists in disliking numerical/probability estimates, preferring to qualitatively discuss “scenarios.” They note one can’t think of all possible relevant events, and point to past numerical estimates that now seem way off.

My talk was on a concrete way to get numerical estimates on extreme risks: refuge futures. I’ve given the subject a bit more thought since I talked on it a few years ago; here is my current concept.

Create a set of underground refuges against disaster, some near major transport access points. For example, a $2 Million shelter can hold 36 people with air, water, food, power for 4 years, at less than $14K per person-year. Near each refuge create a matching resort, which supports a comparably utilitarian lifestyle, but does not protect much against disaster. For example, imagine a cheap hotel near an airport, with a refuge dug below it.

Create and sell transferable tickets representing the right of qualified amateurs to stay in those refuges or resorts on particular future dates. Refuges maintain a multi-year supply of food and power, and are staffed by experts who decide when a disaster justifies sealing it. Qualified folks can use their tickets for a date by showing up at the matching resort; they’ll then be escorted to its matching refuge. Those who are in a refuge when it is sealed remain there until its experts decide to unseal it.

The price of a ticket to a particular refuge on a particular date should vary with the estimated chance of a serious disaster near that date and location. But that price should also vary with other factors, such as interest rates, general wealth levels, the local economy, the total supply of related refuge slots, the risk a ticket holder might fail to arrive in time to use a ticket, and the risk that refuge administrators might not honor valid tickets. How can we disentangle these effects?

Regarding variations in interest rates, general wealth, and local growth, such factors could be roughly corrected for via comparing refuge and resort ticket prices. That is, subsidize a market maker who trades of refuge for resort tickets in some ratio. (Ticket fractions could be a random chance of getting a ticket.) The number of resort tickets required to buy a single refuge ticket could be our key disaster indicator.

While an estimate of how disaster risks vary across space and time would be interesting, it would be far more useful to know how disaster risks vary with events, especially relevant decisions. For example, imagine policy-makers were considering a new geo-engineering program. We could then create conditional tickets, such as tickets to a refuge valid on a date only if this new program was begun by some specified prior date. This would allow folks to trade conditional refuge tickets for conditional resort tickets.

The number of conditional resort tickets required to buy a conditional refuge ticket would be a disaster indicator for that condition. If the disaster indicator was lower given the adoption of a geo-engineering policy than given not adopting it, this would suggest that the geo-engineering policy reduces the chance of serious disaster. The possibility of obtaining such valuable policy info would be a major reason to created this whole refuge-resort ticket system.

Regarding the risk of failing to show up to use a refuge ticket, for each slot available we could sell several tickets at different priority levels. If not all first priority tickets holders showed up, the refuge could randomly allocate slots among those who showed up with second priority tickets. If any slots remained, they’d continue with third priority tickets, etc. We could focus on the total price of all refuge priority level tickets for a date, as that should vary less with variations in the chance folks can’t show up to use tickets.

I’m not sure how best to correct for variations in the local supply of refuge or resort slots. I’m also not sure how best to aggregate trades and prices across diverse resort-refuge pairs.

Added 10p: Regarding the risk that refuge administrators might not honor valid tickets, to get useful prices we only need a substantial chance that tickets will be honored. In order to distort our disaster indicator policy advice, ticket speculators need to expect that the chance of valid tickets not being honored is substantially correlated with chosen disaster policy.  What policies could plausibly create such an expected correlation?

Added 12Dec: I should add that as futures markets in concrete physical services, refuge and resort futures and their derivatives would seem to avoid anti-gambling laws.

GD Star Rating
loading...
Tagged as: ,