Hewitt on Futarchy

Chris Masse taunted, “If he had balls, Robin Hanson would debate Paul Hewitt, instead” of Mencius Moldbug.  Monday Paul posted a 7000+ word critique of futarchy. I commented, “Care to indicate the top three claims you’d most prefer I respond to?” Paul listed three, and Eric Crampton responded much as I would. But since Paul probably wants to hear it from me, here are those claims, and related (long) quotes from Paul’s post:

1. Whether it is possible for (very) long-term prediction markets to be accurate, at the time the decision is made (not 19 or 20 years hence, just before the outcome is determined).

Attempting to forecast total national welfare measure (GDP+) assuming the policy is enacted, which is based on 20 or more years’ of future statistics.  In those intervening 20 years or so, many new policies will be enacted, and every one of them will be expected to improve the national welfare.  What are the odds of such a prediction market being able to accurately (and consistently) predict the actual national welfare that will be determined over a 20 year period?  I’m sure Robin will counter … both decision markets were subject to the same uncertainty about the national welfare measure.  Of course they were.  This only means that both markets must have been equally accurate prior to the policy decision being triggered.  What are the odds?  How could we prove their accuracy?  We don’t have very many long-term prediction markets that can be tested. … David Pennock’s analysis looked at the calibration of long-term markets 30 days prior to settlement.

2. How you expect traders to be able to understand and analyse GDP+ forecasts and the effects of particular policies on this metric, to the extent necessary to be able to improve the accuracy of the market forecast.  – Governments (and teams of economists) have a difficult enough time measuring GDP for the current period, let alone one 20 years from now.

Forecasting national welfare under futarchy is an incredibly complex problem.  I don’t think it is even possible for speculators to make reasonably accurate forecasts of national welfare.  They simply do not possess the knowledge or understanding, let alone a decision model, that would allow them to make accurate predictions.  … Each trader’s prediction is an “accurate” estimate combined with an “error” factor.  The assumption is that the errors cancel out, leaving only the accurate information reflected in the market price.  I think this is likely to be true, but not in every case.  Where the individual errors are large, relative to the known, accurate information, the predicting algorithm is likely to break down.  …  For example, if we were to run a decision market for a policy designed to combat global warming, the forecast would be wildly inaccurate.  The participants simply do not have enough information to make a reasonable forecast.

3A. Why budget constraints and policy adoption ranking was not considered?

Under futarchy, as long as it is clearly shown that a proposed policy would improve national welfare compared with the status quo, the policy is to be adopted.  You don’t have to be much smarter (if at all) than a chimp to understand that no nation would be financially able to implement every policy that met this standard for adoption.  Simply put, there are budget constraints, now and in the future.  All but the simplest of policies involve financial commitments in the future.  Accordingly, policies adopted in the current period will have budget implications in future years, which will limit the ability to adopt future policies that may be proposed (and that should be adopted).  Futarchy makes no mention of budget constraints.


OK, issues #1 and #2 seem to me to be based on two fallacies:

  1. As estimation problems get harder, accuracy quickly falls to zero, after which we literally know nothing about them.
  2. There is some absolute accuracy standard which a governance estimation mechanism must reach to “work.”

Yes, our estimates usually get less accurate further into the future; this happens with pretty much all forecasting mechanisms.  And the academic field of fiance has shown in great detail that this also holds for speculative financial markets; accuracy never falls to zero, but does get worse further out.

There is, however, no absolute “good enough” accuracy standard required to make a forecasting mechanism useful for governance.  A “good” mechanism need only be better than our status quo institutions, and a new mechanism worth investigating need only have a chance of reaching that standard.

Yes, of course it is damn hard to estimate the distant future consequences of policies.  But this is what current institutions must nevertheless try to do.  They do it badly, in part because participants in such institutions have disappointingly weak incentives to look that far ahead.  But they do something, to our advantage.  Our challenge is only to see if we can do better than this situation, via mechanisms that give stronger incentives, even if those are still disappointingly weak.

Yes, our direct comparisons between speculative market estimates and other mechanisms have so far been over relatively short time periods, but the fact that markets have done well there at least suggests we might try them for further futures.  I just don’t understand how you can be so confident of failure as to think we shouldn’t even try.

Issue #3 seems to just be a confusion.   Adopting policies that threaten financial solvency would threaten national welfare, and speculators have incentives to take that into account.

I really think Paul and other critics would do better to focus first on the simpler example of a fire-the-CEO market for publicly traded firms, where many of these issues can be more easily addressed.

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • http://timtyler.org/ Tim Tyler

    Prediction markets typically seem to be zero-sum games.

    Most investors would presumably prefer to play the stock market – where they can expect returns roughly in line with inflation.

    If we want to use prediction markets for serious purposes, perhaps we should invest in them, to provide a bit more reward for the traders.

    Otherwise, trader estimates may be noisy – due to lack of interest.

    • http://hanson.gmu.edu Robin Hanson

      Tim, speculative stock trades are zero-sum. I think you are referring to the fact that bookies often tax bets via not paying interest on deposits. Futarchy bets should of course pay interest.

      • http://timtyler.org/ Tim Tyler

        Yes: investing escrowed bet amounts resolves my complaint – though it may introduce some more parties getting a cut of the transactions.

        Another related issue is market sluggishness. Investors often want to make a buck quickly – whereas prediction markets can take ages to pay out.

  • Michael Turner

    “If we want to use prediction markets for serious purposes, perhaps we should invest in them, to provide a bit more reward for the traders.”

    An interesting idea. I wonder if “micro-futarchy” might be used in micro-credit lending, to help evaluate different business investments. A fairly small amount of money (by Western standards) could buy a lot of developing-world business innovation.

    • http://hanson.gmu.edu Robin Hanson

      Fixed costs limit application to very small problems.

      • Michael Turner

        Size is relative, though. If the proposal evaluators are themselves living on $2/day, $50 to get a quorum of them to consider your idea woukd actually be a lot of money from their point of view. And you shouldn’t worry about micro-lending making a profit overall, because that’s not really the point of micro-lending anyway. Whether you’re lending to a village business or a village government, the point of a “micro-futarchy” scheme would be to have a mechanism that helps people determine what’s likely to work.

  • http://fasri.net Robert Bloomfield

    Wow, Paul Hewitt let you off easy. Here would be my three questions.

    1. Using Blackwell’s theorem, Joel Demski has proved that you can’t identify an optimal accounting standard (such as a definition of GDP) without reference to the decision-maker’s preferences. Since citizens’ preferences differ, it follows that there is no way to identify a unique optimal metric to serve as the number being predicted in a decision market. Why wouldn’t a futarchy devolve into political battles over security definition? What would be the value of having an accurate prediction of a number that almost everyone would agree is not sufficiently germane to the policy being considered (though they would agree for different reasons)?

    2. Prediction markets don’t generally allow for any surplus gains to trade–one person’s gain is another’s loss. Since becoming informed is costly, under traditional models of rational trading, trading volume would be zero OR the market is populated with irrational traders. So is futarchy founded on the notion that widespread trader irrationality is the engine that results in market prices that are more rational and informative than would be achieved through traditional political processes? If so, how do you address the large volume of research (both theoretical and empirical) that demonstrates that large volumes of irrational trade often keep rational traders from eliminating price errors through arbitrage? (See this for a model and this and this for couple of my own laboratory prediction market experiments in Review of Financial Studies.)

    3. Combining 1 and 2, Why is it reasonable to assume that decision markets are robust to conflicts of interest? In the spirit of 1, what if the metric is GDP, and I believe it will rise slightly more under policy A than B, but my own industry gets hammered under A and benefits dramatically under B. If I engage in strategic trade that distorts market prices, and therefore policy selection, where is the guarantee that others will be in a position to discipline me, and do so symmetrically? Note that this is far trickier than just establishing that markets can discipline traders who are misinformed (which is “all” it takes to answer 2).

    I would add a fourth question as well:

    4. The world is full of prediction markets, but as far as I know, decision markets are restricted to traditional voting–including markets where people put their money where their mouths are, whether in corporate elections or American Idol pay-per-vote settings. What are the most promising opportunities for a decision market such as you describe, and why don’t such markets already exist?

    • http://hanson.gmu.edu Robin Hanson

      Robert, 1. National welfare definition should depend on citizen preferences, and a political process would have to choose and update that definition. 2. The ability to influence policy will attract trading to futarchy markets, in and addition trading can be subsidized. I make no claim of “eliminating” errors. 3. I’ve posted previously on manipulation. 4. Innovations don’t exist before they are introduced.

      • drewster

        3. I’ve posted previously on manipulation.

        And as was explained in the comments, there would be many manipulation problems.

      • http://fasri.net Robert Bloomfield

        Your response to my point 4 suggests you are taking it as a rhetorical device. It actually is a sincere question, especially the first part. There must be some policy that is particularly well suited to this innovation.

    • http://torontopm.wordpress.com Paul Hewitt

      I think your point #1 is another way of saying it is futile to try and develop a metric to describe the collective values held by the citizens. No policy could be expected to meet the collective value definition, yet all current proposals would be decided on this definition. Existing institutions attempt to meet the values of some citizens with one policy, which may conflict with other policies that meet different needs (values) of other citizens. Futarchy over-simplifies the policy making process.

      Regarding #3, Robin thinks manipulators are unable to game the markets. Based on certain assumptions, he may be correct. I just don’t think those assumptions hold up in the real world. It is naive to think that manipulators will lose all their money and stop trying to manipulate the markets. Instead, they will find new and more effective ways to purchase policy.

      #4 is the key question. The problem is the case for their use has not been proven. Sometimes they work, sometimes they don’t. No one seems to want to find out why. It is far easier to be a prediction market-er than a researcher, unfortunately!

      Personally, I think the most promising application for prediction markets is forecasting milestone completion dates for projects (software beta test date, product launch date, etc…).

      • http://hanson.gmu.edu Robin Hanson

        You could similar argue that expected utility theory is futile because there’s no way a human could merge all their diverse considerations into a single utility metric.

      • Michael Turner

        “It is naive to think that manipulators will lose all their money and stop trying to manipulate the markets. Instead, they will find new and more effective ways to purchase policy.”

        And you’re willing to bet that prediction markets wouldn’t be able to predict those manipulations — and assess countermeasure policies — faster than the would-be manipulators could come up with them? OK, how much are willing to risk?

      • http://torontopm.wordpress.com Paul Hewitt

        The amount I would be willing to bet would depend on the assumptions you would impose on these hypothetical markets. If you decide to make the manipulators wear signs indicating the direction and strength of their manipulation, I wouldn’t bet very much at all.

        If the manipulators look like you and me (and all of the other traders), I’ll bet the farm they’ll be able to get away with it. Traders in these markets will have a tough enough time trying to forecast a very complex number. On top of that, you think they will have the uncanny abilities to identify manipulators, figure out approximately how manipulative they are, and calculate the corrective bets? Oh yeah, and do so confidently. I would prefer to confidently bet against such abilities.

  • JP

    I’m kind of curious about a couple things with decision markets.

    1. Instead of voting using some method to get a single measure of GDP+, what if each vote is for a single metric scaled so that it will take on a value between 0 and 1. For instance, if I voted for Unemployment as my preferred metric, the resulting contract would be valued at 1 if average unemployment remains below 5% over the period and at 0 if it’s over 10%, with linear scaling in between. GDP growth, inflation, and mortality rates all could be similarly scaled. GDP+ could then be defined as the total predicted value over all votes. For example, with votes like

    “RGDP growth > 5%” : 10 votes
    “Unemployment < 5%" : 5 votes
    "Inflation < 3%" : 6 votes,

    and a policy prescription predicted to have the following effects:
    RGDP = 6%
    Unemployment = 7%
    Inflation = 12%

    would have a predicted GDP+ of
    RGDP : 1.0 (contract value) * 10 (votes)
    Unemployment: 0.6 (contract value) * 5 (votes)
    + Inflation: 0.0 (contract value) * 6 (votes)
    ————————————————–
    13 (predicted GDP+)

    Would this approach work ?

    2. In general, what are the reasons why the decision market should be correct for the consequences of the counterfactual action (i.e. the one not predicted to work best). If everyone expects that one action will work less well, will a decision market have a hard time evaluating the magnitude of how much worse that action would be?

    3. If the stakes are low enough, does adding a degree of randomization increase the reliability of the estimates of how much worse the counterfactual action would be?

    • http://hanson.gmu.edu Robin Hanson

      JP, 1. There are many possible political procedures for defining a welfare function. For modularity, I considered a proposal to that process as is for now. 2. It won’t be obvious which choice is best for a while. 3. Randomizing policy can have large costs.

  • Peter Twieg

    I should’ve read up on the specific of futarchy a long time ago, and I’m glad that Hewitt’s response provided some incentive to do so. My main concern about futarchy wasn’t touched upon by Hewitt, and I doubt Moldbug’s critique raises it either, so either it’s relatively unobvious concern or it’s relatively stupid one. I’ll hope for the former.

    I found myself more worried about the non-linear effects of policy combination on GDP+. If there are N policy dimensions, then in a prediction market over the effects of varying any one policy dimension, then insofar as policies become better or worse in combination with other policies, traders will have to take into account the expected policies that will be enacted in those policy dimensions in order to generate their expectations about how different values in a single dimension of policy turns out. One could have prediction markets over combinations of policies across several dimensions at once, but obviously we can’t even begin to feasibly have markets over a significant fraction of the entire combinatorial space of different policy dimensions.

    Thus if policy prediction markets (or any predictions markets, I guess) are not very independent… well, this is where it gets hard for me to stab at what would happen. I guess the basic concern would be that you’re no longer comparing a given policy change against the status quo, but against a market estimate of what other policy changes would be enacted. And these estimates seem to be recursive and messy to me and would require an absurd amount of knowledge to work properly. I’m sure this has come up before, so I’m curious if there’s a more concise formulation of my concerns, as well as a response to them.

    • http://hanson.gmu.edu Robin Hanson

      This is already a standard issue in policy approval in all organizations, and for low rates of change is dealt with adequately by evaluating and approving changes one at a time. For moderately higher rates of change combinatorial markets exist and should deal with this problem.

    • http://torontopm.wordpress.com Paul Hewitt

      I believe I did touch on this issue in my full post. Essentially, traders are trying to forecast the effect of the current policy and all other future policies likely to be enacted (under the present and all future budget constraints). Of course, this is impossible. But, under futarchy, it is equally impossible to make the forecast for the status quo.

      Futarchy makes decisions based on the difference between the two forecasts. Unfortunately, there is an implied assumption that both decision markets are equally accurate. Not only do I question the accuracy of these markets, I question the equality of their forecast accuracy.

      I believe the problem is so complex that it would be impossible for decision markets to make accurate forecasts other than by sheer luck. Hardly the basis for rational decision-making.

      • http://hanson.gmu.edu Robin Hanson

        I make no such implied assumption.

      • http://torontopm.wordpress.com Paul Hewitt

        Robin, if you aren’t assuming that very similar decision markets, with potentially the same traders, trading at the same time, are likely to be equally accurate, the question becomes, why not? This speaks to the consistency of prediction market forecasts.

        If both markets aren’t equally accurate (more or less), what can you say about the accuracy of the difference, the forecast policy effect?

      • http://hanson.gmu.edu Robin Hanson

        I make as few assumptions as required, and I don’t see why this assumption is required. Do you need to assume this about our status quo institutions to recommend them?

      • http://torontopm.wordpress.com Paul Hewitt

        No, we don’t make this assumption about our status quo institutions, but they don’t automatically trigger policy implementations either.

        Effectively, futarchy hard-wires the entire policy decision to the forecast effect of the policy on the national welfare measure. All of the decision analysis is encoded in the welfare measure. Garbage In, Garbage Out. That metric had better be good.

      • http://hanson.gmu.edu Robin Hanson

        In a constitutional government, there is no escaping something being “automatic”. We could “automatically” implement any bill passed by Congress. Or if Congress is subject to court review then we “automatically” do what the court says. There has to be some highest rule, and then we “automatically” do what that rules says. In any case the quality of our policies is determined by the quality of the estimates on which we base our policies. How “automatic” are those policies has little to do with it.

  • josh

    You deserve a huge amount of respect for debating Moldbug, though I suspect you will be talking past each other. His best argument against futarchy seems to be that its implementation would violate the laws of political science and is thus impossible (his other arguments weren’t really any good). You generally seem unconcerned with this, and focus on ‘We should have futarchy, because it would be really good if we did’ (as well it would be). These two points arguments are not in conflict and I wonder if, upon careful reflection, either of you would disagree with th others position.

  • http://mchouza.wordpress.com Mariano Chouza

    The list of “design issues” in your futarchy paper is really comprehensive!

    I think that the most likely failure modes are the ones you titled Welfare Measurement Might Be Corrupted and Easy to Measure Values Would Be Over-Emphasized, though the current system is hardly immune to them. The only way to avoid the divergence between GDP+ and “quality of life” is competition between political entities, IMHO.

  • http://torontopm.wordpress.com Paul Hewitt

    I’m a bit disappointed! Yes, I did wish to hear your thoughts about my concerns, but you spent most of your allotted space citing portions of my post.

    I responded to Eric Crampton’s post, here.

    This is your response to my first two issues:

    Issues #1 and #2 seem to me to be based on two fallacies:

    A.As estimation problems get harder, accuracy quickly falls to zero, after which we literally know nothing about them.
    B.There is some absolute accuracy standard which a governance estimation mechanism must reach to “work.”

    I am well aware of these issues. I never said, nor did I assume, that accuracy falls to zero. I did not imply (at least I hope I didn’t) that we “know nothing” in such cases.

    I did not indicate that futarchy decision markets (or any prediction market for that matter) must meet some absolute standard for accuracy. Therefore, you have not really answered my questions.

    A “good” mechanism need only be better than our status quo institutions, and a new mechanism worth investigating need only have a chance of reaching that standard.

    I agree with you that a better (I wouldn’t say “good”) mechanism needs only to be better than the status quo one, to be worthy of investigation. However, “better” has not been proven (even on paper). “Worthy” needs to be considered in terms of costs and benefits. I grant you that the potential benefits of futarchy might be immense, but the likelihood of success is infinitesimal, leading to a likely low expected benefit. The costs could be huge, depending on how many bad decisions are made. Wouldn’t it be better to reform the negative aspects of our existing institutions? (a good example can be found at the very end of my post)

    You have still not answered the question regarding long-term “accuracy” of prediction markets.

    Yes, our direct comparisons between speculative market estimates and other mechanisms have so far been over relatively short time periods, but the fact that markets have done well there at least suggests we might try them for further futures. I just don’t understand how you can be so confident of failure as to think we shouldn’t even try.

    As you know, I have a problem with your definition of “done well”, when used in the same sentence with “prediction markets” and “accuracy”. There have been very few marginally successful prediction market applications. These have all involved, not just “relatively” short time periods, but actually, very short time periods. In most cases, the time periods were so short, that there was no opportunity to use the prediction for decision-making. Such prediction markets are trivial pursuits.

    We both know that uncertainty increases the further we are away from the actual outcome. This counters your argument that short term “successes” suggest we should at least try longer term markets. Rather, they suggest that if short-term markets are not significantly better than status quo institutions, they are even less likely to be useful for longer term predictions.

    David Pennock’s anaysis of ideosphere.com long-term prediction markets shows they were calibrated 30 days before the outcome was revealed. In no way does this imply that the markets were calibrated at earlier times. It is possible that they were, but we may never be able to determine this. And, even if they are calibrated far in advance, the flat distributions don’t provide useful information for decision-making, they tell us we need more information or the outcome is too uncertain to predict. Either way, it does not support the potential of futarchy.

    The true test is the expected net benefit of “trying” (investigating further) futarchy. I think logic and practicality trump faith. Perhaps I am too confident, but just because a chimp might hit the bulls-eye doesn’t mean we should point him at the dartboard.

    With respect to the budget constraint issue, you endow speculators with rather remarkable attributes! Not only must the “informed” be economic seers, they must also be excellent accountants (bodes well for me). I should mention that I thought of this possibility, but dismissed it out of hand. So few people can manage their own budget let alone one for the entire nation. If you aren’t convinced, now, perhaps you will be in a few weeks when the credit card bills arrive!

    • http://timtyler.org/ Tim Tyler

      Re: “Whether it is possible for (very) long-term prediction markets to be accurate, at the time the decision is made”

      It seems pretty irrelevant. What matters is whether prediction markets are better than other methodologies for making predictions.

      • Peter Twieg

        Yep, I think this is the essential point which Hewitt is failing to address. If he doesn’t think that prediction market accuracy falls to zero, but only to a low level…. does he believe this is better or worse than the usual political processes at estimating long-run policy effects? If he thinks it’s better, why?

      • http://torontopm.wordpress.com Paul Hewitt

        Of course, we all agree that the real issue is whether prediction markets are better than other methodologies for making predictions. My point is that the case for prediction markets has not been made, at all. There is a tiny bit of proof that they are as good as alternative methods, and in a very few cases, very slightly better. Also, you need to be aware that even the slightly better prediction markets had the benefit of the alternative forecasting institution available to it. That is, the official forecasters at HP were also participants in the ever-so-slightly better prediction markets.

        If we were to look at the present policy making institution and conclude that it is almost totally broken, making far more bad decisions than good, even I might conclude that futarchy should be given a chance (only because we would have nothing to lose). But that isn’t the case, is it? Certainly, the status quo institution is broken, but it’s not that broken.

        Almost all very long-term prediction markets would exhibit very flat forecast distributions. This means that, even if they were “accurate” (in the calibration sense), the prediction would be very wrong, very often, and we would have no idea when this might occur. I put it back to you (and Robin) that there must be an even better methodology than the status quo or futarchy.

      • http://hanson.gmu.edu Robin Hanson

        Since you emphasized how very hard is long term forecasting, have you any argument to offer for why you think our current institutions do such a good job, relative to how badly you think market speculators would do?

    • http://hanson.gmu.edu Robin Hanson

      You seem to think that our only relevant data is papers with “prediction markets” in the title, rather than the much larger literature on speculative financial markets. And of course it is “not been proven”; almost nothing is proven about new proposed mechanisms.

      • http://torontopm.wordpress.com Paul Hewitt

        I’m sure there is a whole new chapter (or two) that will be written on speculative financial markets. If you have particularly important references that support the case for prediction markets and their use in futarchy, please let us know.

        In one of your papers on futarchy, you mention that “experts” often do have relevant information that might have stopped governments from enacting bad policies, but that such expert advice is usually obtained to support a predefined position, instead. I agree with you.

        Rather than reinventing an entirely new institution, why not reform the existing one to make better, proactive, use of experts, directly?

  • http://fasri.net Robert Bloomfield

    I think the problem is far more severe than that. Prediction markets may be a cost effective way of getting a prediction. But as I understand it, Futarchy goes beyond prediction by asking voters to accept the following position:

    PROPOSED: Law X will be enacted without amendment if the market price of Security S is greater than P at date D, and will not be enacted otherwise.

    The proposal is empty if no one can agree on an X they are willing to leave unamended, or if they cannot agree on a definition of S, or more generally if the proposal is not renegotiation-proof (that is, if a traditional vote overturns the market outcome at date D). The proposal is damaging if voters would want to overturn the outcome but are somehow restricted from doing so.

    I can’t imagine a setting in which such a proposal would pass, especially if it included a restriction against renegotiating the outcome.

    • http://hanson.gmu.edu Robin Hanson

      I propose no restrictions on amendments – at any time any proposal could be approved, including proposals that amend previous proposals.

      • http://fasri.net Robert Bloomfield

        Quoting directly from your paper (p13), “When an approved betting market clearly estimates that a proposed policy would increase expected GDP+ (E[W|N] > E[W|Q]), that proposal immediately becomes law.” The “without amendment” is implied, and I only made it explicit.

        If you are saying that there will be no restrictions limiting future changes, then the futarchy proposal will never have any lasting effects on policy.

      • Peter Twieg

        If you are saying that there will be no restrictions limiting future changes, then the futarchy proposal will never have any lasting effects on policy.

        I don’t understand why this would be, except to say that policies will be continually refined in order to better-maximize GDP, which would generally be considered a good thing.

  • ERIC

    To me the key of the “markets vs. institutions” debate is incentives. I’m not sure re-election incentives are as important as post-presidency book/speaking tours because you were likable (Clinton, but the list is long I’m sure) and did popular things however bad they turn out for the public as judged by history.

  • rob

    “Investors often want to make a buck quickly – whereas prediction markets can take ages to pay out.” — Investors can try to make a buck quickly in long-term prediction markets in the same way they can short-term trade commodities futures markets. I don’t believe anyone is saying you must hold contracts until maturity.

  • ravi hegde

    “Rather than reinventing an entirely new institution, why not reform the existing one to make better, proactive, use of experts, directly? ”

    Because it is clear that short term social status related benefits have a very real chance of skewing our perspective or altering our behaviors as regards the future. Prediction markets (just like financial spec markets) will perform better on the average but would also fail spectacularly on occasion. Prediction markets also be be inefficient in small numbers just like bulletin board stock markets.

    Paul’s ideas are tantamount to questioning the efficacy of financial speculation markets.

    • http://torontopm.wordpress.com Paul Hewitt

      Did you mean the efficiency of financial speculative markets? If so, I’ll take the heat.

      Your statement “Prediction markets … will perform better…” is without support. Also, when they do “fail spectacularly”, there is, presently, no understanding as to why. Inaccurate markets look identical to accurate ones.

      To blindly make decisions based on markets that will sometimes be right, sometimes wrong (and we have no way of knowing when), is just plain crazy. You might have an argument, if prediction markets (for long-term forecasts) are proven to be more accurate than alternative forecasting methods. Unfortunately, there is no such evidence, yet.

      • http://hanson.gmu.edu Robin Hanson

        Do you have access to some forecasting method whereby one can see ahead of time the difference between correct and incorrect forecasts?! If no such method is available, how can it be crazy not to use one?

      • http://torontopm.wordpress.com Paul Hewitt

        No, there is no way of knowing in advance. This places added importance on being “right” most often. In Enterprise Prediction Markets, we can obtain some information to make this assessment.

        If it turns out that the distribution of forecasts is too flat, it may be that it is not possible to forecast that particular metric (perhaps by any known method). Maybe the only alternative is to abandon that metric and choose another (or others) that can be measured and forecast to some reasonable level of accuracy.

        Related to the issue of accuracy, how does futarchy handle the case where the decision markets “clearly” indicate a positive policy effect, but the prediction market distributions are very flat?

      • ERIC

        “…there is…no understanding as to why” … is this the narrative fallacy at work here? I think there are plenty of explanations for a failure but agreement is limited.

        Isn’t the point that a robust prediction market would incorporate “alternative forecasting methods”? To me, the point of a market price is that it incorporates “everything”. Prediction markets share/uncover information quite well, don’t they?

        If you think company XYZ is worth $20/share and the market value is $10 or $30, doesn’t that give you pause to think maybe you’ve missed something? If nothing else, is this not a major benefit of any type of market…you are forced to view a reality you may not see otherwise given your biases?

        Isn’t the hedging benefit of markets worth something too? Unless hedging is not important to you at all. If it is, how could it be done easily outside of a market?

      • Hal Finney

        Maybe it would be interesting to consider a hybrid of futarchy and conventional legislatures, to deal with Paul’s “flat distribution” problem. As I understand it, the issue is that markets in practice may be unable to penetrate the fog of uncertainty regarding the semi distant future, and therefore cannot effectively differentiate the expected long-term effects of various policies. This leads to market prices that are equal for various proposals, to within some noise term, preventing effective policy action.

        Imagine that we had the usual futarchy rules that clearly superior policies are implemented automatically (ie policies where the expected improvement exceeds some noise threshold). But, in addition a legislature would be permitted to enact policies which made things no worse, again to within a noise delta. If it does turn out that in some policy areas, the market ends up scratching its collective head when it tries to predict the future, the legislature would then be able to take action.

      • http://hanson.gmu.edu Robin Hanson

        Hal, I fear that would allow the legislature to do pretty much anything they wanted, as long as they did it in small enough steps.

      • Hal Finney

        If the legislation took things too far then a futarchy proposal to reverse their recent moves would rise above the noise and win. But then maybe you’d have to have some extra rule to prevent this back and forth pattern wastefully repeating. It gets pretty complicated, and we don’t know if the flat distribution would be real or not.

  • Pingback: Debate is raging between Robin Hanson and the futarchy critics | Midas Oracle .ORG

  • http://hanson.gmu.edu Robin Hanson

    Hal, on reflection the minimal change and recursive elements of futarchy already deal with this problem of what to do when markets are silent. Presumably the new futarchy regime would start out with all the old rules, plus the one new rule that markets can set all policy. Legislatures would thus continue to be able to pass bills that change policy until markets approved a specific futarchy proposal to take this power away from legislatures. If such a proposal passed, it would be because speculators expected that the harm from continued legislature policy changes would outweigh its benefits.

  • http://torontopm.wordpress.com Paul Hewitt

    I have posted a concluding comment on my blog, here:

    http://torontopm.wordpress.com/2010/01/04/the-future-of-futarchy/#comment-148

    Robin, thank you for continuing the discussion. Good luck in The Debate!

  • http://torontopm.wordpress.com Paul Hewitt

    In response to your comment on my blog…

    I agree with most of your comments, Robin, particularly as they relate to financial markets. However, the types of markets contemplated under futarchy differ from financial markets in terms of their informational efficiency (at least that’s my opinion).

    I’m not saying that the information accuracy drops to zero, but I am saying that it drops to a very low, or at least an unreliable level. Consequently, for the long-term markets, the market aggregates a lot of guesses, as opposed to reasoned opinions.

    Perhaps we differ in our beliefs about market efficiency of decision markets.

    In terms of distinguishing between innovative ideas worth trying and others, I draw the line with a reasonable cost-benefit analysis. Rather than have faith that a market will “work”, even when logic tells you it won’t (a la Miracle on 34th Street), I need to know there is a logical reason why it should (or at least could) work. Then, if the potential benefits are likely to exceed the cost of trying, give it a try.

    Maybe, we need to take a closer look at prediction market efficiency.

  • Pingback: Futarchy and its discontents | weychi.com