Reply to Moldbug

Concise.

I wonder if you would agree with MM that crazy ideas like yours don’t face any genuine intellectual hostility within academia.

Robin Hanson, you have fallen into an affective death spiral around prediction markets. Somewhere you went from believing that prediction markets work for certain things, with certain types of participants, under certain circumstances, to believing that prediction markets always work.

The problem with prediction markets is that they create a financial incentive to spread disinformation. Consider this interview,
http://www.thedailyshow.com/video/index.jhtml?videoId=221517&title=jim-cramer-unedited-interview
Starting at about 7:50, Jim Cramer, who was a hedge fund trader, admitted to successfully spreading false rumors about Apple in order to capitalize on changes in its stock price. Earlier in that interview, he says that sort of thing is banned, but that the ban is impossible to enforce. If predictions behave like the stock market did, then the net effect of creating a prediction market will be to create one piece of good information, at the expense of promoting disinformation everywhere else. I do not consider that an acceptable trade-off.

If you play in the stock markets you can expect returns in line with inflation. Prediction and decision markets are zero-sum games though – and thus seem likely to be fundamentally less attractive. There, the people most likely to want to play are those with a vested interest in the outcome.

Moldbug is confused about insider trading. Insider trading isn’t illegal because the traders would be unfairly more informed than the other market participants. It’s illegal because the shareholders who put the management of the company in place require that the management disclose any information they have to them for free before they trade on it.

http://www.vnunet.com/vnunet/news/2209899/hacker-keep-profits

A Ukrainian hacker stole information from a company and traded on it. The only crime he committed was breaching in a computer system, NOT insider trading although he obviously had access to privileged information.

Generally though, I am also skeptical of futarchy. For example, I am not convinced that one can devise good tests for policies. If the goals are too specific metrics, horrible thing will happen (many policies guarantee low unemployment, most of them are BAD). If they are too general (say people report they are happy), the effect of a specific policy may be too diluted by other policies to allow a reliable payoff.

I also find the idea of voting on values a bit repulsive.

Oh, and I missed this:

You can ask for a full probability distribution. If speculators know they don’t know anything, then they will give you a broad distribution that expresses a lot of uncertainty. This is them telling you they don’t know much. In your nose example, they may just give you the distribution over nose sizes for elite Chinese.

This suggests to me that it’s been a while since you read Feynman. My fault – I should have been clearer. The entire point of Feynman’s story is that no one knows anything about the Emperor of China’s nose. It’s a gedankenexperiment, not a problem in physical anthropology.

My worst-case scenario was a market in which no one knows anything, but no one knows that no one knows anything. This is a pure noise market. Moreover, since no one knows it is a pure noise market, no one will design it to generate a full probability distribution.

The cut-off sentence: “a decision market is a prediction market which is used to make decisions, ie, through a mechanical linkage as in futarchy.”

And I also missed

For nations I suggest a full national welfare; GDP is only a first step to such a measure.

You would, wouldn’t you? Have you considered the King of Bhutan’s little innovation, “Gross National Happiness?” I’ll bet you’re already a fan:

http://www.nytimes.com/2009/05/07/world/asia/07bhutan.html

I’m quite confident that you have no idea at all of why anyone would consider the idea of constructing a national goodness index, and governing to maximize it, in any way, shape, or form bizarre, Orwellian, or just plain… retarded. And that’s okay. Perhaps that’s how it should stay.

If not, I suspect you also skimmed the Carlyle excerpt at the end of my post. If you reached it. If you tire of responding to me, for which I can certainly excuse you in advance, I’m sure your readers would enjoy a response to Carlyle. Chapter 2, “Statistics,” in the pamphlet on Chartism.

Morons and non-morons, which I often call sheep and wolves, are not distributed independently across topics. The wolves are consistently attracted to the sheep, and the net effect of having more wolves and more sheep is to have more accurate markets. Having a decision depend on a market induces people who want to manipulate the price, who are in effect sheep, since they attract wolves. You are somehow worried that decision markets that determine government policy would attract sheep without wolves, but I can’t imagine why.

In plain English: the initial state of any market is a mix of accurate and inaccurate predictors. The accuracy of the market depends on the proportion of sheep to wolves, which the designer of the market does not control. Therefore, the designer cannot promise accurate initial results.

In a futarchy, which is a market in which government decisions are mechanically linked to the results of a prediction market, accurate initial results are essential, because the random initial mix of sheep and wolves will guide initial decisions. I can’t imagine how this could be acceptable.

Moreover, a futarchy will contain a third class of animal: bears, animals that are willing to lose money by making intentionally bad bets because they profit from bad policies. Your explanation of why the results of a futarchy will not be dominated by bears is that the bears will lose too much money to the wolves, and be unable to continue. Over time, this may be true, but it is unlikely to be true in the initial state.

Thus, prediction markets differ from futarchy in that the operator of a prediction market can afford to take its results with a grain of salt. As I’m sure your customers do.

Moreover, you seem to be using an oversimplified model which assumes, unjustifiably, that the errors made by sheep follow a normal distribution – whose center is the just price as defined by the wolves. I hope I am reading you wrong here, but if not I urge you to abandon this assumption, which is obviously unjustified by reality.

Let me put this discussion slightly in context by explaining the difference between a trained market, one that has undergone adaptive selection, and an untrained market, eg in the initial state.

Trained markets (such as Wall Street or Vegas) achieve remarkable results because, over generational periods of time (think of how long people have been betting on stocks, or on the NBA) they foster a remarkable quantity of expertise. They breed super-wolves, who feast on the sheep. The market is not magic. It does not create this information. It merely aggregates it. It is the humans who are the experts.

Untrained markets aggregate the opinions of their participants, just like trained ones. The aggregation step is not what makes the market expert – the expertise is. A market mechanism works perfectly well as a way of voting on probabilities, but voting is all it is. Both trained and untrained markets are useful – but they are very different things.

It is all too easy to confuse these very different things, and associate the remarkable expertise that collects in a trained market with the simple betting mechanism shared by both. Once a logical mind has made this mistake, it is very easy to come up with ideas such as futarchy.

>> Now, consider an alternate version of the experiment, in which the professor says: “Class, I have a jar of jellybeans under my desk. How many jellybeans are in it?” That’s a market in which no one knows anything. Don’t expect an accurate aggregate.

This is an experiment we can do. Would you like to refine it before betting against an accurate aggregate? Hint: you would.

Moldbug-

In a futarchy, which is a market in which government decisions are mechanically linked to the results of a prediction market, accurate initial results are essential, because the random initial mix of sheep and wolves will guide initial decisions. I can’t imagine how this could be acceptable.

You could start the decision market as a prediction market. At first people would bet on whether policy A will have results X. Once the wolves have eaten the sheep, and the market has learned to make very accurate predictions, you can then hook the market up to the machine and have the bets actually dictate the policy.

Your other points are spot on though. There are very few decisions that can be defined as a multiple choice problem with a well defined result. And if you could trust the leaders to phrase the questions, the choices, and the result measurements, you could trust them to use the prediction markets as a tool, rather than a machine.

here’s a question…

does the informed bettor make a bet on what he actually believes the correct answer to be? or does he place a bet to make the aggregate more accurate?

example: I believe the correct answer to be 0, someone else has bet -1. Do I bet 0 or 1?

of course this hinges on where the bettor is making his profit. Does he profit from a correct outcome or does he profit from being correct regardless of outcome?

does the informed bettor make a bet on what he actually believes the correct answer to be? or does he place a bet to make the aggregate more accurate?

The informed bettor makes a bet when her personal odds for the outcome differ from the market-implied odds. For example, if her personal odds are 4 to 1 against and the market is offering 8 to 1 against, she will take the proffered bet. Note that she’s betting against what she believes the final outcome will most probably be. Supposing she is accurate, over the long run she will lose four such bets for every one she wins, but when she wins she’ll make back twice the money she lost.

The wolves are consistently attracted to the sheep, and the net effect of having more wolves and more sheep is to have more accurate markets.

Maybe, but the noise trader literature shows that this is not a slam-dunk. No, you don’t have to include every disclaimer, but I think this is important (though not relevant to MM), so I’m including it.

Most of this is completely irrelevant.

According to Wikipedia, under futarchy, “elected officials define measures of national welfare and prediction markets are used to determine which policies will have the most positive effect”.

The biggest, most obvious problem comes *before* the “and”. Most of this discussion is about what comes after it.

The idea of a single, standardized measure of welfare, forcibly applied to a large population, is idiotic, whether it’s GDP, GHP, or anything else. It leads inevitably to disaster.

Abramowicz (author of “Predictocracy”) felt similarly to RP and so wanted to depend on legislators to determine after the fact whether policy was good or not. Hanson explained why he preferred just having them define GDP+ here. There is little reason to think that Congress will do a good job of that. Right now however they are both defining goals and deciding implementations.

RP, futarchy can be applied at small scales, such as running firms using stock price as the outcome measure.

Professor Hanson,

Your sentences continue to be information-packed. Let me take them one by one:

Moldbug, the relevant task of speculators in a firm stock market changes every time the firm changes its policies or its competitive environment changes. Does this mean that such stock markets are always in their “initial state”?

An excellent question. It goes to the heart of the frequentist probability interpretation.

Remember the Lakers-Nuggets game. What sample space is it in? The space of professional-sports games, the space of NBA games, the space of Lakers games, the space of Nuggets games, the space of playoff games. And of course it is also a one-time event: itself.

Each of these events but the last consists of a set of different events. Patterns can appear in this set. A successful prediction algorithm

Each sample space has its own wolf/sheep classification. A bettor may be an expert on the Lakers, for example, but know nothing about the Cavaliers. Her predictions may be reliable for playoff games, but not for regular-season games. She may understand the NBA, but know nothing about the NFL.

As a non-abstract matter, for example, the “wolves” in a healthy financial market (I’m not convinced that ours is entirely in that category) are people who have some insight on the general task of mapping from the publicly-released accounting data of a corporation, to its future stream of returns. This is a sample space which is clearly defined and obviously generalizable across.

If you send these wolves to Vegas and have them set odds for NBA games, you have a new initial state. Same if you have the bookies start a hedge fund.

And speculators who deal with a series of similar policy proposals are in a similar situation, so that their experience with previous proposals is good enough.

Sure. (Although of course it depends on what you mean by “policy” – this is a much broader sample space.) However, you still face the problem of training the market in the first place!

And your theory that “initial” markets statues have useless prices would seem easy to test in real financial market data – have you done such tests?

First, it’s not easy to test – because testing it would imply creating a new financial market. Moreover, our present financial markets have been trading more or less continuously for centuries, and any new financial market is a financial market, ie, in the same general sample space of the old. It would therefore attract the wolves of the old, rendering the market quite efficient and the experiment quite useless.

But moreover, even if such a test could be performed, it would be entirely irrelevant to the analysis of futarchy. It may, or may not, be the case that some process of selecting initial participants in financial markets, or NBA markets, or jellybean markets, any other markets, will produce a population of sheep and wolves whose aggregate guess is good. But again, skill in one sample space is unrelated to skill in another sample space by definition.

So, even if you could create a new financial market and you did, whether or not its initial aggregate guess was quite accurate, the result would imply no conclusion at all about the quality, distribution or motives of the initial participants in a policy decision market. Thus the experiment is useless.

Moldbug, you say every new financial market we have ever seen hasn’t really been different enough to see the effect you postulate, that “new” markets start out with a long learning period when they have useless prices. But somehow you think the markets I propose would be different enough to see this effect, and that it would be intolerable to suffer such a training period, even though markets can tell us how uncertain they are, and we could run the markets in an advisory-only capacity for as long as we wanted until we saw a clear enough track record to inspire our confidence in gradually relying on their advice more. We already have some relevant lab data, and we could try the concept first in smaller organizations before applying them to larger organizations. But apparently you think none of this could be good enough because we could never trust any track record about the past to predict anything about the future.

I like many of Moldbug’s points, but I do have a problem with the Emperor of China example.

This blog has previously discussed probability distributions. If I ask you what my father’s name is, and you’ve never heard of him, “I don’t know” is a quick answer but not a technically complete one. You actually do know quite a bit about my father’s name: you know it’s more likely to be John than Cindy, and more likely to be either of those than Zeeeeeeeee. I could offer you a series of bets about my father’s name at different odds (would you take a bet where I give you a million dollars if my father’s name is Mike, and you give me one cent otherwise? What about a bet where you give me fifty cents if my father’s name is Zeeeeeeee, and I give you three dollars otherwise?) and we would discover that you actually had some quite strong opinions on how likely it was that my father’s name was various different things.

Moldbug seems to be a frequentist, which makes this a little harder to explain, but hopefully he will agree that there is a certain probability distribution on which he stands to make the most money in a series of bets regarding my father’s name. That distribution will probably be something like the percentages of males born in my father’s generation who have various names: a set along the lines of [2% John, 1.5% Mike….001% Ezekiel…00000001% Zeeeeeeee.]

We can come up with a similar distribution for any unknown question. For example, I am 99% confident that the Emperor of China has a nose less than a foot long, and almost 100% confident that he has a nose less than a mile long. My distribution for Gormball would be very close to 50-50, but if you offered me a fair bet in favor of Neptune and against Ganymede, I’d take it: Neptune’s so much bigger than Ganymede that its alien population would probably be much bigger (I wouldn’t have expected aliens to inhabit a gas giant, but this question seems to presuppose they can) and a higher population means more potential athletes and therefore a better team (the same reason I’d take a bet that the US could beat Andorra at any sport you choose). I wouldn’t jump at the chance to bet on Neptune’s superiority, but if I had to make a choice, I could honestly say I give them a >51% chance. We should have some probability distribution for all questions, even if it’s the boring case of “equal probability for all alternatives”.

An ideal prediction market (and Moldbug has done a good job of proving that real world ones are not ideal) should be able to feed us a better probability distribution than any other method on all questions, including the Emperor of China’s nose question. Even if it feeds us an all-alternatives-equal distribution, we’ve still learned something useful; we’re now too smart to fall for someone saying the answer is definitely X and we can know that with certainty. The person who knows that there’s a 50% chance the market will rise tomorrow and a 50% chance it will fall tomorrow is better off than the guy who’s been tricked into believing some scam that claims they can predict the market for him.

If all possible methods would do poorly (for example, the Emperor of China is by an astounding coincidence a mutant freak with a five mile long nose and no one could have anticipated this) then prediction markets will also do poorly. But assuming we do need a best guess on the Emperor’s nose length for some reason, then as far as I can see prediction markets won’t do any worse than anything else.

@RH, MM, respectively:

“And your theory that “initial” markets statues have useless prices would seem easy to test in real financial market data – have you done such tests?

First, it’s not easy to test – because testing it would imply creating a new financial market. Moreover, our present financial markets have been trading more or less continuously for centuries, and any new financial market is a financial market, ie, in the same general sample space of the old. It would therefore attract the wolves of the old, rendering the market quite efficient and the experiment quite useless.”

The CDS market was pretty awful at calculating proper zero-sum insurance premiums for about its entire existence until the massive risk repricing of last year. Here you see a new product where not enough historical data exists to properly calculate a fair price – this plays to MM’s point, that such things take time to “train”- we never went through a true balance sheet/credit crisis until last year, and the CDS market was making idiotic predictions because it had never “learned” what happens in one of those.

RP, futarchy can be applied at small scales, such as running firms using stock price as the outcome measure.

Robin, are you saying you no longer advocate futarchy as a form of government, as you did here?

Moldbug, you say every new financial market we have ever seen hasn’t really been different enough to see the effect you postulate, that “new” markets start out with a long learning period when they have useless prices. But somehow you think the markets I propose would be different enough to see this effect…

Yes, because the problem of analyzing corporate profits is substantially different from the problem of analyzing public-policy outcomes. How many people do you know who are expert in both?

it would be intolerable to suffer such a training period, even though markets can tell us how uncertain they are, and we could run the markets in an advisory-only capacity for as long as we wanted until we saw a clear enough track record to inspire our confidence in gradually relying on their advice more.

If I’d seen this proposal in your futarchy paper, I might have given you credit for it. Perhaps it’s there and I just overlooked it. But yes – see my two objections above.

One: as soon as you switch the market from advisory to binding, the state of the market changes, because a “bear” strategy (losing money in the prediction market, to purchase decisions which make even more money) becomes viable. Nothing at all prevents a wolf from turning into a bear.

Two: I think market validation is a lot more difficult than you think. Let me reason through a series of points, and you can tell me the first one you disagree with.

A: To validate the efficiency of a market is to show that no counterexample trading strategy can be designed which profits against it.

B: Given any historical record for any market, it is trivial to design a counterexample strategy which profits in hindsight.

C: Most such profitable counter-strategies amount to data mining/snooping/dredging, and will not exceed random performance in future. A profitable counter-strategy is legitimate only if it profits across the entire sample space, ie the space of all possible markets – including future as well as past.

D: Distinguishing between legitimate and illegitimate counterexamples to the claim that a market is efficient (a) requires an understanding of the problem that the market is solving (ie, the entire sample space, including both past counterfactuals and future hypotheticals), and (b) involves an assessment that is qualitative, not quantitative (data does not help).

E: Therefore, to decide that the market is efficient, we must (a) prove a negative (no valid counter-strategies exist), and (b) must use qualitative assessments to filter out invalid counter-strategies.

F: Therefore, the skill needed to decide whether to turn the market on demands expertise in the problem which is at least comparable to the expertise that would be required to do without the market at all, and instead make the decisions directly.

Note, as a gut-check confirmation of this reasoning, that people still argue furiously over whether the stock markets are efficient or not. If there exists some methodology that could validate the efficiency of a market, you’d think it would have been deployed already.

So, no: I haven’t seen your data. However, if I was shown your data, it would put me in the position of asking: who do I believe, (a) my brain or (b) your data? If I go with (a), I am at worst consistent in my error. Thus I have no logical interest in your data. If you are right and I am wrong, you have a better chance of explaining the error to my brain, than intimidating me with data.

Alternatively, if you are willing to agree that there is an important qualitative distinction between a trained, adapted or iterated prediction market and an untrained one, this is my main substantive point. While, for the reasons above, I don’t feel that modifying futarchy according to Devin’s proposal would make it a good idea, I’m willing to admit that this improvement rescues it from “retarded.”

And, of course, we’re in agreement that prediction markets are very good things and should be much more common than they are. However, I suspect that your private-sector customers would become more, rather than less, confident in the value of these devices if you erred more on the side of overstating their limitations. CEOs respond very well to underpromising and overdelivering.

I suspect your readers would also appreciate (a) a comment on the vitality of the adversarial process in the modern university, and/or (b) a comment on the demand for mechanical decision procedures in government (ie, a response to Carlyle).

My two cents:

Futarchy is useless because it essentially boils down to a newer weirder freakier form of American democracy. If the market is large and everyone can play and have a say, then this is just another election where people get the dumb crap they deserve. Imagine America going nuts betting over “Will gay marriage destroy our families?” There cannot possibly be wolves in such markets; only demagogues would gamble, since the outcome is nothing more than measuring who’s got more demagogues. This is like democracy. Sure, Professor Hanson would like us to believe that futarchy wouldn’t be deployed for such cheap stunts; but then again, Democracy was never supposed to be deployed for such cheap stunts either.

If the decision concerns a small specific narrow interest, then the special interests will manipulate the market in order to get the policy decision they want. This is like lobbying.
When the question of going to Iraq comes up, the people who care the most manage to bribe enough politicians, journalists and talking-heads to assure the war happens. In a futarchy, they’d spend this money on bending the market’s decision their way. Concentrated benefits and dispersed costs give the beneficiaries the ability to lobby and get their way. I see no reason why this would not translate to prediction markets. So long as the economic consequences of a decision exceed the market cap, then this is going to happen.

And so, through votes and money influencing decisions, we would eventually pretty much get the same outcome as we do today with lobbyists and democracy.

This exposes the problem with government decisions in general. No matter how great or accurate or wonderful or futuristic the decision-making process (dictatorship, democracy, monarchy, futarchy, populist) government decisions are a bad idea.

And this is why libertarians like Dr. Hanson would make far more sense if they stuck to arguing for liberty and curtailing governments rather than trying to find more elaborate ways of squaring the circle of making government decisions intelligent and good. There is a perfect tried and tested way of utilizing the wonderful knowledge of markets to make people’s lives better: real life actual markets without government intervention.

Thanks for your response, Mencius. Yes, inflation is a better example. But I still don’t understand why you’re critiquing prediction markets for being unable to predict really difficult questions well when traditional methods also fail and the markets might do a little better.

Do you believe that traditional methods (like asking a famous economist) are consistently more likely to be right than ideal prediction markets? If so, why would a policy-maker be more likely to choose this brilliant economist than prediction market investors?

Do you believe that there’s a high chance that even well-trained, highly-developed prediction markets will be far from ideal, and that it will be hard to detect when they’re far from ideal, and that therefore a famous economist can consistently beat a real prediction market if not an ideal one?

Do you believe no one should ever try to predict difficult questions? Even in a libertarian society, in a few cases policy makers will just need some idea how likely certain things are. For example, the military will need to know what the chances are of getting involved in different types of wars over the next decade. This is a good example of one of your difficult problems where there are probably a lot of false signals, but the military can’t just shrug and give up and not plan for anything; it needs a best possible guess, even if it knows that guess has a huge margin of error. Someone somewhere in the Pentagon is probably working off of a vague belief like “X percent chance we’re in a land war with China over the next decade; Y percent chance we’re in a naval war.” Why not get X and Y from a prediction market instead of pulling it out of some general’s rectum? Neither will be all that good, but the market ought to at least be better.

Or do you believe that prediction markets are as likely or more likely to be right than famous economists, but that policy-makers will have the good sense to understand that a famous economist is probably wrong, whereas they’ll trust a prediction market blindly? That’s a pretty reasonable point, but if education were at all possible it would be better than giving up on the whole idea forever.

By the way, I like your blog. I think. I’m still too overwhelmed to really be sure.

Ultimately, markets are only as good as actual experts are.
Have you read What does a free market require? A market can function with mindless actors. Galton’s “wisdom of crowds” (though not actually a market) did not depend on any single individual having an accurate guess. Regarding political predictions, are you familiar with Tetlock’s “Expert Political Judgment”? The experts don’t do that well compared to a simulated monkey throwing darts at a board. Bryan Caplan gives a commendable defense of experts in his review of the book (which you can read here) but it’s mostly against a populist misinterpretation of Tetlock’s results. If you simply try to select “the best” experts you are probably going to wind up overpaying the tournament winners, and still have a small sample size (with differing opinions within the sample suppressed by group-think) subject to variance error and overconfident in its own predictions. Furthermore, selecting the best experts requires competence in the selectors to know expertise when they see it.

If some things are completely non-predictable then markets will not successfully predict outcomes. Unfortunately, they don’t produce a result that says “this price is the result of people making guesses as to what other people think about the underlying”.
As others have mentioned, in the emperor’s nose issue bettors will note the uncertainty by giving a wide probability distribution. Markets in which there is more confidence will concentrate probability in a narrower area.

Of course, there are a variety of prediction markets already in existence. And, though I do not often quote the Bible, it is apt to point out that “ye shall know them by their fruits.”

One that is particularly apposite to the present discussion is that in credit-default swaps. Continuing in the same mode, one is tempted to recall the passage “for now is the axe put unto the root of the trees, so that every tree that bringeth forth not good fruit is hewn down, and cast into the fire.”

Moldbug, you do not understand what you read. Our conversation is over.

Note that if the decision is welfare-diminishing compared to the status quo, there will be a group of people losing more money than the bears gain. They would have a strong incentive to buy the decision back, wouldn’t they?

They have the same incentive to buy the decision back as the consumers do to buy an open market in foreign sugar. Unfortunately, corn producers and sugar producers have stronger incentive to buy the market closed.

Concentrated benefits, diffuse costs.

Whether they’re “accurate” depends in large part on definitions of accurate, and who controls those definitions. They’ll be different in decision markets (where accuracy depends on policy outcomes) than prediction markets (which simply predict, or not, what outcome will occur).

We face this today and I do not believe we have *truly* objective criterion for evaluating whether the implementation of a policy on any subject of import the least bit controvercial was accurately predicted. That is to say, we already have a lot of data on policy implementations and outcomes, and it’s much harder to agree on whether their effects were accurately predicted in advance than with a market that simply predicts whether A or B will happen (“Will Barak Obama’s Stimulus Package pass and be signed into law” is a much simpler question than “what will the effect of Stimulous Package A be, will it be better than Stimulous Package B, and will either be better than doing nothing, and over what time frame, and with or without other ripple effects” – CBO’s predictions, for example, are considerabily different than the White House’s).

In the abstract, people propose Policy A, others say Policy B will be better, some say the status quo is better than either. The advocates of Policy A say it will cost $100 Billion Woolongs and create 1.21 gigajobs. Critics say that it will cost four times that much in the end and produce half as many jobs.

Policy A is enacted, it ends up costing several multiples of what was predicted and net job growth is undetectable. Advocates say it was because it was underfunded at the beginning, and that other factors impacted it, and given how the economy performed, conditions would have been worse without it: It saved jobs. Critics say their criticisms were accurate and that the policy made things worse ultimately, particularly because of unintended consequences (Brawndo consumption went up 400%, causing crop failures and an increase in illiteracy and out of wedlock childbirth, and these things aren’t included in the advocate’s data regarding whether the policy outcome was good or bad). Adocate’s rejoider is that well B, C, and D happened in the interval as well, and without Policy A in place, those things would have excacerbated the situation, so A really saved us from catastrophy.

This is much more open to manipulation than a prediction market in that sense alone.

Also, I think Hanson is well too dismissive of the ability of “Bears” to manipulate outcomes. In a world where George Soros & friends could cause a run on the Pound, it’s not hard to foresee that a subset of interested parties who will benefit disproportionately from the implementation of a policy will have more interest in investing enough to make sure it happens than everyone else. Will “Wolves” follow that money, because it is more likely to occur (and thus pay out), or will they vote with the “sheep” (those whose direct interest in the policy choice is less strong, and indeed who may pay the costs and receve no benefit)?

“Smart Money” is more likely to go with the “Bears” on this, in my opinion. Their market behavior will be based on an evaluation of what will “win” rather than which will be the best policy to implement, and everyone else in the market will then be “sheep” (AKA the “marks”).

For some reason a lot of the exchanges on “Wolves” vs. “Sheep” reminded me of Phil Helmuth going on rants about “idiots” who have offsuit Queen-Ten and raise. Phil’s “idiots” win hands, get to final tables, even win tournaments, despite the fact that though over time – if your time horizon is long enough – the pros do much better than “dead money”. When interest and passion is high, a lot of sheep enter the market who otherwise might not be there – and this will make the outcomes different than in the smaller-scale running experiments Hanson foresees as testing and building the case for Decision Markets.

But in this situation the way the sheep will be separated from their money by the wolves is not directly tied to which policy outcome was accurately predicted to be preferable. The decision market will be dominated by those “Wolves” who are most skilled in knowing which decision will dominate, not which one is good policy and which one(s) are worse, and by players with an interest in the outcome (Bears) regardless of it’s overall impact (the “I’m All Right, Jack” set). The wolves will prosper in the market from investing correctly in which policy alternative will win, not which one will have the most benefitial impact.

It “may” (to use Professor Hanson’s word) be that those don’t correlate at all. They “may” even be opposed. I’m not sure I’d bet my life that they will be alligned. I agree with Moldbug that there is no good way to test it – I have little confidence that “experimenting” in a small scale on inconsequential policy decisions won’t produce the same outcomes that going big on major choices will. Wolf money will tend to follow, rather than counterballance, Bear money, and everyone else by definition will become Sheep and get shorn.

So in other words, no change.

To those who imagine something important happening when the market is “switched” from advisory to mechanically binding, think about the way a U.S. president is elected. Early presidential elections clearly didn’t mechanically translate popular votes into a certification of a victor. Has the electoral process become mechanical now? If so, when did it become mechanical? If not, how many people are actively working to make it more mechanical and do you have any reason to expect futarchists to expend effort making futarchy more mechanical than the electoral college currently is?

Do excuse the necro-posting, but I was startled to discover no-one’s even touched on what I feel is the weakest link in the futarchy proposal.

First, let me just say that prediction/futures markets are awesome, and Hanson’s motivation — that thinking up effective and economic public policies ought to be profitable too is a laudable one.

With that said, the whole system falls at the first hurdle, when we are required to vote on the way of evaluating the outcome of public policies. Not only does it assume that
1. there is some objective “value function” for evaluating public policy, but it assumes that —
2. whilst democratic vote is deemed a poor way of working out the outcomes of public policy, it is a good way to work out what those outcomes should be

Assume, for instance, that I am a fundementalist Mormon. For me, an important aspect of public policy is that it minimises drug use and permits bigamy.

I am a minority in the US, so my democratic votes are not going to have that much effect on the value-function. Whether the decision markets work or don’t, I’m going to be pretty dissatisfied with the outcome.

However, if we abandon the idea of a universal value-function, we may also abandon the controversial decision market for predicting public policy outcomes. I “shop” for a legislature in the same way I shop for baked beans; by their outcomes. If a legislature’s public policy provides poor outcomes — intended or not — I will choose another.

The exact method of “shopping” for a legislature may vary. Moldbug’s Patchwork, Friedman’s PDAs, or even the old-school Panarchy are all adequate solutions.

This, in my opinion, is a far better way to make inventing good policy profitable.