Finding “Our” Beliefs

If only you are interested in a topic, you’ll have to think it through for yourself, or pay someone else to think on it. But if many folks are interested in your topic, you might hope to share the thinking work with others.

Some social institutions seem to serve this “our beliefs” function. Today you can see if your question is answered in wikipedia, you can search a library for answers in respected books or journals, and you can call up an expert credentialed in a related area to see if they know of an answer.

Of course these institutions are imperfect. Calling experts is expensive, quick searches only find some of the many differing opinions out there, and encyclopedia answers, while unique, only address a limited range of questions. If you find an answer you know is wrong, it can take a lot of work to change it. You might have to devote a whole career to the attempt, and even then you might not be rewarded for the effort.

Ideally, we’d want an “our answers” institution with easily-found accurate answers on as many questions as possible. Hopefully, anyone could ask any question, and answers would be consistent with each other and across time. If incentives to give accurate answers were strong enough, we might even let anyone correct any answer.

Prediction markets might allow such a better answers institution. Ordinary financial prices offer consistent unique answers that anyone can fix, but for a typical ordinary question, it is very hard to figure out which price combinations might answer it. In contrast, prediction market questions can be expressed in simple ordinary language.

If (money-based) prediction markets were legal, anyone could add a new question for a modest fee (<$100), and quickly get unique answers consistent with all other questions. Anyone could fix any answer, and would have incentives to do so accurately.  Or anyone could pay to make any answer more accurate. So far, tests have found prediction markets to be consistently as or more accurate than other prediction institutions with similar resources.

Of course ordinary prediction markets do have one big limitation: they only directly answer questions that eventually become clear for other reasons. But this allows more than it might seem. For example, because we will later know which candidate is elected by what margin, and how big is the post-election unemployment rate, prices today can say which candidate is expected to most help unemployment if elected.

This example suggests one way to expand the range of questions prediction markets can cheaply answer: start with a set of base questions, and then let users ask and answer questions from the vast space of combinations of those base questions. For example, starting with a base consisting of all the specific future readings of all weather stations, users could ask most any weather question of interest, such as whether this next winter will be colder where they are living now, or in the particular city where they are thinking of moving.

In my next post I’ll talk about a big advance my research group has achieved in the implementation of such combinatorial prediction markets.

GD Star Rating
loading...
Tagged as:
Trackback URL:
  • Ely

    Have you had much experience with the Stack Exchange family of sites? I frequently use and contribute to the ones on computer programming, math, and computational statistics. All mostly just for fun. I’ve even seen some publication-quality answers on such sites, often with excellent bibliographies. And topical surveys are excellent (I recall a particularly awesome one on the different senses of stationarity in stochastic signal processing at the DSP stack exchange). I am curious whether others believe collaborative outsourcing of marginal tech problems along research paths will play an important role in the near future. How soon (if ever) before a competent professor can reduce need for graduate assistants due to something like stack exchange? How widely would such an effect be felt? Would it “free up” grad students to do more theoretical tasks or just simply mean there are fewer grad students period? (Or have no effect?)

    • adrianratnapala

      But also Stack Exchange is still an experiment.  We don’t know whether it only works for certain fields, and we don’t even know if these (excellent) sites will continue to work.  For example, I find Stackoverflow less useful for me than it was just one year ago.  This might just be a blip, or it might because it has grown so large that it is hard for potential answerers to find my questions.

  • V V

    What happens if I ask a self-referential question like “This prediction market will fail to predict the answer to this question.”

    • DanielLC

      Then it’s not clear whether or not it succeeded, and thus you’d get the same result as if you asked something like whether or not God exists.

      • V V

         If you add a deadline the question becomes well defined.

    • Charles R. Twardy

      🙂   We use questions like that as practice questions at http://daggre.org.  Some interesting differences in behavior for various variants on self-referential.  See the “practice” category. 

  • Leo Shine

    What stops people from asking questions to which they have private knowledge of in order to make money off people who only know public knowledge?

    • Jayson Virissimo

      Nothing, but by doing so, they are increasing the accuracy of “public knowledge” by causing price changes.

    • http://overcomingbias.com RobinHanson

      It costs money to ask a question, money that goes to the people who answer the question. So in this scenario you’d just pay yourself.

      • V V

         Doesn’t that create an incentive to keep knowledge secret in order to answer questions?

        Sure, when you answer correctly, you reveal part of your knowledge, but that’s just the few bits of the answer, compared to the complexity of the background knowledge that you might have used to compute the answer.

      • http://overcomingbias.com RobinHanson

        I didn’t claim that all prediction markets cause everyone to immediately reveal all their info.

      • V V

         But could prediction markets incentivate people to withhold knowledge that they would otherwise reveal more easily?

      • http://overcomingbias.com RobinHanson

        Maybe if we didn’t let farmers sell their food, they’d give some of it away.

      • V V

        Bad analogy:

        1) Information can be copied, food can’t.

        2) Without prediction markets, you can profit from sharing your theoretical knowledge, by writing books, giving lectures, writing academic papers, and so on. My concern is that prediction markets could incentivate keeping theoretical knowledge secret and only selling predictions which can’t be easily used to reconstruct the underlaying theory.

      • Leo Shine

        I’m sorry I’m still confused, why is the potential gain limited to lower than the money paid for asking the question? If I had private information that I slowly released as trading is happening can’t I predict how the market is going to respond to these releases and then make the right trades just before I release the information to profit. Why can’t a strategy like this one recover more than the original sum?

  • http://twitter.com/PaulSHewittCA Paul Hewitt

    Using the weather question, doesn’t that presuppose that it is possible to make an accurate prediction?  There are many, many important questions that we would like to predict, such as future weather, climate change, earthquakes, volcanic eruptions, etc…  Unfortunately, it simply is not possible, given our current state of ignorance, to make reasonably accurate predictions, no matter how “expert” one might be.

    What if the base questions cannot be predicted, given the existing level of knowledge and information?  Then, it doesn’t make much sense to devote a lot of effort trying.  Predicting climate change might be the best example of this type of scenario.

    Still, I am hopeful that combinatorial prediction markets can be used in some areas to make much better predictions than can be made through any other model.  Predicting project completion dates comes to mind.

  • Pingback: Overcoming Bias : We Can Do Low-Treewidth Combinatorial Prediction Markets!