My Best Idea: Decision Markets
Many (Poincaré 1908, Schumpeter 1911, Ogburn 1922) have said that, as there are so many good ideas out there, most innovation is just simple combos of prior good ideas. This seems true of my best idea.
April 25, 1996, thirty years ago today, I first posted my best idea: decision markets, i.e., speculative markets that advise specific decisions by estimating decision-conditional outcomes. A.k.a., “futarchy” as applied to governance. It’s not my deepest, grandest, beautiful, or hardest won insight, just the one with the biggest expected impact.
My idea was a simple combo of two other long-well-known ideas.
The first prior idea I built on is that speculative markets do quite well at aggregating info. This was explored in theory (Emory 1896, Gibson 1889, Bachelier 1900) and in data (Cowles 1933, Working 1934). Even so in 1996, US regulators in practice only allowed risk-hedging, not info aggregation, as an “economics rationale” to allow markets to exist. (The allowed “price discovery” rationale was tied to helping other markets hedge risks.)
In 1984 I left grad school in physics and philosophy of science at U Chicago to go to Silicon Valley to do AI research, and on the side work with Xanadu, trying to invent the World Wide Web. Around 1988 I first started to have doubts about the Xanadu vision of reforming public convo by making criticism easy to find, and wondered what else we could do instead. So I started to think and write about the big potential of making speculative markets to aggregate info on far more topics. Like most everyone who first enters this space, I was first thinking mainly in terms of markets on the usual topics we see in mass media, punditry, and public policy debates.
The second prior idea I built on is that info is mainly valuable by informing specific decisions. For many centuries we’ve seen calculations of the value of certain specific info for specific decisions. And then we developed more general theory (Ramsey 1928, Hosiansson 1931, Blackwell 1951, Savage 1953, Schlaifer 1959). At Caltech social science grad school 1993-1997, I learned decision theory and the standard value of info calculation. Then wondering where speculative markets could add the most info value, ~1996 I realized that this would likely come from markets estimating specific outcomes given specific decision choices.
As I was one of the first to write on the big potential of prediction markets, many who entered this space over the years approached me. At which point I usually pitched this decision market concept. Which usually pushed them away, as they were focused, as I was initially, on those mass media and punditry topics. But I have doggedly persisted.
Most all innovations combine simple elegant ideas with messy details that make those ideas work. Mine is no different. To find the right messy details, one needs concrete trials and experiments trying different detail versions. It has been hard to find orgs willing to do this, as org decision making is usually quite political. But in the last few years we’ve thankfully started to see some trials.
As an econ professor who specialized in governance, I can assure you that the world is greatly structured by the fact that we typically have pretty incompetent governance. Imagine a governance that, when assigned a goal, would reliably achieve that if it is in fact feasible. This would radically reshape our whole world. (Yes, even if we soon get powerful AIs.) As decisions markets plausibly enable such competent governance, this is why I estimate their expected impact to be so very great.