Tag Archives: Prediction Markets

10% Less Democracy

My GMU econ colleague Garett Jones has a book coming out in February: 10% Less Democracy: Why You Should Trust Elites a Little More and the Masses a Little Less. I just read it, and found it so engaging that I’ll respond now, even though Jones’ publisher surely prefers book publicity nearer its publication date.

Regarding to the vast space of possible governments, it seems to me that Jones uses “more democratic” to describe situations closer to a 100% democracy ideal, wherein all citizens have an equal say and can vote directly on all government choices, with government able to control all other choices. In this framing, anything that makes it harder for voters to simply and directly choose the options they understand and prefer makes a system less democratic.

That includes electing representatives instead of directly voting on policy, and also logrolling, divided government, and other complexities that make it harder for citizens to tell what is going on and to assign responsibility. It includes any limits on who can vote, and any ties to outsiders that limit internal discretion, like treaties with other nations or selling debt to bondholders. And it includes longer terms for the elected, and more indirection, such as when politicians appoint other officials instead of directly electing those other officials.

By these standards, our current system obviously deviates greatly from a fully democratic ideal, and Jones approves of most of these deviations, especially ones that result in longer term views and in more informed voters and officials. And he’d like to move modestly further in such less-democracy directions, though not too far, as he accepts that strong autocrats tend much more to kill their citizens, allow famines, and create more economic growth volatility (though similar average levels of war and growth). Jones musters a lot of data in support of his modestly-cut-democracy view.

I did a few surveys yesterday which suggest that overall my Twitter followers find the existing degree of democracy pretty close to their ideal, though a majority would also prefer a reduction. So, for them, Jones’ position doesn’t seem at all controversial:

In the past I’ve tended to think about all this in terms of principal-agent problems. It doesn’t always make sense to make all decisions yourself, if you can instead consult an agent who does or could know more than you. But you must be careful to keep such agents under sufficient control. So if they are careful, voters may reasonably gain by delegating to experts. However, the reason I found Jones’ book so engaging is that I found a lot of the data Jones presented to be challenging to understand from this principal-agent view. (And also, it was a pleasure to engage such fundamental issues.)

For example, politicians with longer terms but without safe districts act at the end of their term more like politicians who have shorter terms. They pass fewer bills, make more pork projects, more trade protection, more labor market regulation, more environmental reforms, have optimistic budget forecasts, and support fewer currency devaluations. Apparently, voters don’t remember much of what politicians do beyond the last years or so.

Cities with appointed (vs elected) city treasurers pay 0.7% lower interest rates. Central bankers who are more independent produce lower inflation and fewer financial crises, at no overall cost to unemployment or real growth rates. Elected judges give more awards to in-state folks at the expense of out of state folks, and their legal opinions are less often cited as precedent. Nations with more independent judges have stronger property rights, less red tape to start a business, fewer employment regulations, and less government ownership of banks.

In general, elected regulators allow utilities to pass fewer costs on to customers, resulting in both lower prices but also in less investment and worse service. Electric utilities regulated by elected officials have lower consumer prices, pay higher interest rates, and more blackouts. Elected telecom regulators oversee lower capacity services, and independent telecom regulators gave in less to demands by government telecom organizations.

Jones is inspired by these examples to support Alan Blinder’s proposal to create an independent central-bank-like expert body to set tax policy, with Congress deciding only broad parameters like total take, progressively, and corporate fraction.

Some of these patterns can be understood in terms of commitment problems. When there is a temptation for politicians to renege on prior commitments, it can help to let them commit via choosing appointees who are out of their control at the crucial moments of temptation. Commitment problems seem especially important for city treasurers, central bankers, and utility regulators. And law court decisions are a classic commitment problem.

These results can also be somewhat understood in terms of the advantages of retrospective relative to prospective voting, and of aggregation in retrospective voting. That is, if voters are impatient and can better judge how their life has gone in the past than they can judge the effects of policies on the future, then voters can be better off when politicians are judged more on their past accomplishments, which happens more with longer terms. And if voters find it hard to attribute responsibility to specific officials, it can be better if they they focus on electing fewer bigger politicians (like mayors) who appoint more other officials.

However, I’m not sure that commitment problems and retrospective voting actually account for most of these patterns. Jones’ book subtitle talks instead about trusting elites, and do note that there is a much more widespread pattern of governments authorizing high status experts in each area to decide key results in their area, including who are to be considered the next generation of experts.

Consider how much we defer to military experts on defense, police on crime, medical experts on health, academics on research, lawyers on law, etc. Yes, in principle we could punish them if past outcomes in their area were bad, but we rarely do this. And professional licensing is a more general policy by which government authorizes control by the high status people in each area. These policies seem less like clever indirect ways to commit or to enable retrospective voting, and more like a simple status effect, wherein voters and politicians want to be seen as respecting and not opposing those high in status.

While all these examples that Jones didn’t include seem to be examples of less democracy, they seem to me to less clearly support his position that this kind of less democracy is good. Excess professional licensing does a lot of harm. The military seems to overemphasize things that high status leaders like more, like fighter planes and aircraft carriers. Medicine seems to overemphasize high status doctors over other medical professionals. Education and research seems to overemphasize the topics by which academics gain the highest status. Law seems overly complex and to overemphasize the need for expensive lawyers. And so on.

Compared to arguing over specific policies, I very much appreciate Jones calling our attention to larger more general issues regarding the design of our political system. But I prefer to generalize even further, via something like futarchy. I can support futarchy without needing opinions on whether tax policy should be run by a panel of independent experts, nor even whether it is in general better or worse to let high status experts in each area control those areas. As long as we use some reasonable (broad retrospective) national welfare measure, with futarchy I could instead trust a general mechanism to make good choices about such things.

GD Star Rating
loading...
Tagged as: , , ,

Ways to Choose A Futarchy Welfare Measure

In my futarchy proposal, I suggest a big change in how we aggregate info re our policy choices, but not in how we decide what outcomes we are trying to achieve. My reason: one can better evaluate proposals that do not change everything, but instead only change a bounded part of our world. So I described choosing a “national welfare function” the way we now choose most things, via a legislature that continually passes bills to edit and update a current version. And then I described a new way to estimate what policy actions might best increase that welfare. (I also outline an agenda mechanism for choosing which policy proposals to evaluate when.)

In this post, I want to consider other ways to choose a welfare function. I’ll limit myself here to the task of how to choose a function that makes tradeoffs between available measured quantities. I won’t discuss here how to choose the set of available measured quantities (e.g, GDP, population, unemployment) to which such functions can refer. Options include:

1) As I said above, the most conservative option is to have an elected legislature vote on edits to an existing explicit function. Because that’s the sort of thing we do now.

2) A simple, if radical, option is to use a “market value” of the nation. Make all citizenships tradeable, and add up the market value of all such citizenships. Add that to the value of all the nation’s real estate, and any other national assets with market prices. With this measure, the nation would act like an apartment complex, maxing the net rents that it can charge, minus its expenses. (A related option is to use a simple 0 or 1 measure of whether the nation survives in some sufficient form over some very long timescale.)

3) A somewhat more complex option would be to define a simple measure over possible packages of measured quantities, then repeatedly pick two random packages (via that measure) and ask a random citizen which package they prefer. Then fit a function that tries predict current choices. (Like they do in machine learning.) Maybe slant the random picks toward the subspaces where citizen choice tests will add the most info to improve the current best fit function.

4) An option that requires a lot of complexity from each ciziten is to require each citizen to submit a welfare function over the measured quantities. Use some standard aggregation method to combine these into a single function. (For example, require each function to map to the [0,1] interval and just add them all together.) Of course many organizations would offer citizens help constructing their functions, so they wouldn’t have to do much work if they didn’t want to. Citizens who submit expensive-to-compute functions should pay for the extra computational that they induce.

5) Ralph Merkle (of Merkle-tree fame) proposed  that “each citizen each year report a number in [0,1] saying how well their life has gone that year”, with the sum of those numbers being the welfare measure.

I’m sure there must be other interesting options, and I’ll add them here if I hear of some. Whatcha got?

A common issue with all these mechanisms is that, under futarchy, every time a bill is considered, those who trade on it acquire assets specified in terms of the then-current national welfare measure. So the more often that the official welfare measure changes, the more different kinds of assets are in circulation. These assets last until all the future measures that they refer to are measured. This is a reason to limit how often the official measure changes.

Inspired by a conversation with Teddy Collins.

Added 22Aug: Some polls on this choice:


The status quo approach is the most popular option, followed by market value and then fitting random picks.

GD Star Rating
loading...
Tagged as:

Decision Markets for Monetary Policy

The goals of monetary policy are to promote maximum employment, stable prices and moderate long-term interest rates. By implementing effective monetary policy, the Fed can maintain stable prices, thereby supporting conditions for long-term economic growth and maximum employment. (more)

Caltech, where I got my PhD in social science, doesn’t have specialists in macroeconomics, and they don’t teach the subject to grad students. They just don’t respect the area enough, they told me. And I haven’t gone out of my way to make up this deficit in my background; other areas have seemed more interesting. So I mostly try not to have or express opinions on macroeconomics

I periodically hear arguments for NGDP Targeting, such as from Scott Sumner, who at one point titles his argument “How Prediction Markets Can Improve Monetary Policy: A Case Study.” But as far as I can tell, while this proposal does use market prices in some ways, it depends more on specific macroeconomic beliefs than a prediction markets approach needs to. 

These specific beliefs may be well supported beliefs, I don’t know. But, I think it is worth pointing out that if we are willing to consider radical changes, we could instead switch to an approach that depends less on particular macroeconomic beliefs: decision markets. Monetary policy seems an especially good case to apply decision markets because they clearly have two required features: 1) A clear set of discrete decision options, where it is clear afterward which option was taken, 2) A reasonably strong consensus on measurable outcomes that such decisions are trying to increase. 

That is, monetary policy consists of clear public and discrete choices, such as on short term interest rates. Call each discrete choice option C. And it is widely agreed that the point of this policy is to promote long term growth, in part via moderating the business cycle. So some weighted average of real growth, inflation, unemployment, and perhaps a few more after-the-fact business cycle indicators, over the next decade or two seems a sufficient summary of the desired outcome. Let’s call this summary outcome O.  

So monetary policy just needs to pick a standard metric O that will be known in a decade or two, estimate E[O|C] for each choice C under consideration, and compare these estimates. And this is exactly the sort of thing that decisions markets can do well. There are some subtitles about how exactly to do it best. But many variations should work pretty well. 

For example, I doubt it matters that much how exactly we weight the contributions to O. And to cut off skepticism on causality, we could use a 1% chance of making each discrete choice randomly, and have decision market estimates be conditional on that random choice. Suffering a 1% randomness seems a pretty low cost to cut off skepticism.

For more, see the section “Monetary Policy Example” in my paper Shall We Vote on Values, But Bet on Beliefs?

GD Star Rating
loading...
Tagged as:

Understandable Social Systems

Brennan and Magness’ book Cracks in the Ivory Tower: The Moral Mess of Higher Education reviews many ways that colleges overpromise, and fail to deliver. It confirms (with Caplan’s Case Against Education) a picture wherein ordinary people are pretty clueless about a big institution in their lives. This cluelessness also seems to apply to many other life areas, such as medicine, charity, politics, etc. In each area, most people don’t seem to understand very basic things, like what exactly is the product, and what are the incentives of professionals?

That is, we each live in many complex social systems, such as political, transport, medical, religious, food, and school systems. Due to our poor understanding of such systems, we have low abilities to make intelligent personal choices about them, and even worse abilities to usefully contribute to efforts to reform them. This suggests a key criteria for evaluating social systems: understandability.

When we don’t understand our social systems, we can be seen as having little agency regarding them. They are like the weather; they exist, and may be good or bad, but we are too ignorant to do much about them. If a situation is bad, we can’t work to make it better. Some elites might have agency re such institutions, but not the rest of us. So a key question is: can we reform or create social institutions that are more understandable, to allow ordinary people to have more agency regarding the institutions in their lives?

One possible solution is to use meta-institutions, like academia, news media, or government regulators, that we may better understand and trust. We might, for example, support a particular reform to our medical system based on the recommendation of an academic institution. Our understanding of academia as a meta-institution could give us agency, even when we were ignorant of the institutions of medicine.

As an analogy, imagine that someone visits a wild life refuge. If this visitor does not understand the plants and animals in this area, they might reasonably fear the consequences of interacting with any given plant or animal, or of entering any given region. In contrast, when accompanied by a tour guide who can advise on what is safe versus dangerous, they might relax. But only if they have good reason to think this guide roughly shares their interests.  If your guide is a nephew who inherits your fortune if you die, you may be much less relaxed.

So here’s a key question: is there, at some level of abstraction, a key understandable institution by which we can usefully choose and influence many other parts of our social world? If we think we understand this meta institution well enough to trust it, that could give us substantial agency regarding key large scale features of our social worlds. For example, we could add our weight to particular reform efforts, because we had good reasons to expect such reforms to on average help.

Alas, academia, news media, and government regulators all seem too complex and opaque to serve in this key meta role. But three other widely used and simpler social mechanisms may be better candidates.

  1. Go with the majority. Buy the product that most other people buy, use the social habits that most others use, and have everyone vote on key big decisions. When some people know what’s best, and others abstain or pick randomly, then the majority will pick what’s best. Yes, there are many topic areas where people don’t abstain or pick randomly when they don’t know what’s best. But if we can roughly guess which are the problematic topics, then in other areas we may gain at least rough agency by going with the crowd.
  2. Follow prestige. Humans have rich ancient intuitive mechanisms for coordinating on who we find impressive. These mechanisms actually scale pretty well, allowing us to form consensus on the relative prestige of nations, professions, schools, cities, etc., and via these proxies, of individuals. Related ancient mechanisms let us form consensus on elite opinion, i.e., on what prestigious people tend to think on any given topic. Yes, elites are biased toward themselves, and to express opinions that make them seem impressive. Still, we can do worse than to follow our best.
  3. Embrace Winners. Nations, cities, firms, professions, teams, media, clubs, lovers, etc. often compete, in the sense that some grow at the expense of others that shrink or disappear. Often they compete for our personal support. And often we see judge that the competition is roughly “fair” and open to many potential competitors. In such cases, we may embrace the winners. For example, we may try many competitors, and stick with those we like best. Or we may go with the lowest price offer, if we can control well enough for quality variations.

Each of these big three mechanisms has limits, but they do seem to satisfy the requirement that they are very simple and many ordinary people can at least roughly understand why they work, and where they run into problems. Together they may cover a pretty wide range of cases. In addition, we can augment them with many other approaches. For example, we can just expose ourselves to choices and follow our intuitions on which are best. We can follow choices by those we know and trust well, those who seem to know more about a topic, and those who seem more honest in their evaluations. Together all these tricks may give us substantial agency re the social institutions in our lives.

Yet those examples of how badly most people misunderstand school, medicine, etc. suggest there is vast room for improvement. And so I look for ways to do better. Not just at designing institutions that actually work, in the sense of producing efficiency, equity, generality, robustness, evolvability, etc. Not just at designing meta-institutions with these features. And not just at gaining the support of majorities or elites, or at winning many fair competitions in the world. I seek meta-mechanisms that can also be simple and clear enough to their advantages be understandable to many ordinary people.

This is the context in which I’d like you to see my highest hopes for prediction markets. I offer them not just as mechanisms that actually work, producing and aggregating info at a low cost. After all, there may be other complex and subtle mechanisms that experts expect to achieve similar or even somewhat better results. But the problem in that case is that ordinary people may wonder how well they can trust such expert judgements.

No, I’m interested in the potential for prediction markets to serve as a simple understandable meta-institution, on par with and at the level of going with the majority, following prestige, and embracing winners. Simple enough that many ordinary people can directly understand why they should work well in many applications, and also to understand roughly where their limitations lie. Yes, not everyone can understand this, but maybe most everyone could know and trust someone who does understand.

GD Star Rating
loading...
Tagged as: , , ,

Space Fund

At a space conference this last weekend, I was inspired to ponder the key problem I see regarding space colonization: how to recruit the great passion among so many to support and participate somehow in the topic today, while avoiding the vast waste that most likely results when that passion is directed to greatly premature near term projects.

Someday humans will colonize Antartica, the top of the Himalaya mountains, and the bottom of Earth oceans. But this won’t happen until these colonies are in the ballpark of cost-effective relative to more familiar locations. Quirky preferences or religious devotion can make a modest difference, but can’t overcome huge cost differences.

The same applies to colonization of space, a place much harder to colonize. While extra passion and quirky preferences can make a modest difference, mostly space colonization just can’t happen until near when it would be feasible given more ordinary motives. Efforts spent well before that time are mostly wasted, unless they are especially well targeted toward easing later efforts when such colonization is nearly feasible.

Here’s my decision-market idea for tying current passion to useful future efforts:

  1. Create a space fund that passively reinvests its assets to grow over a long period, a fund to which anyone can donate,
  2. Define an ex-post measure of successful space colonization. For example, LNYD = Log of number N people living in space for at least Y years by date D.
  3. For a modest fee, let anyone at anytime submit a proposal for how to spend the entire space fund. Any proposal is fair game, including transferring all of this fund to a new fund managed a new way.
  4. Create financial assets $LNYD that pay in proportional to this measure LNYD. (This may require setting a min & max value for the measure.) Let people trade these assets for cash, creating a LNYD market price.
  5. Each proposal submission is evaluated via a LNYD-based decision market. That is, for each proposal, on a particular unique pre-announced date, market speculators may trade LNYD assets for cash, in trades that are called off if (or if not) this proposal is approved. If the LNYD price difference between approval and non-approval is clearly positive, the proposal is approved. (The price difference threshold used here should reflect the fact that this system should reject a great many proposals, and approve only one.)

Under this system, people today who want to feel involved with space colonization can do so in three ways: 1) donate to the space fund, 2) develop and submit proposals for approval, or 3) trade in the markets that decide if to approve proposals. Later, when space colonization is nearly feasible, so that money spent can actually make a difference, these decision markets should make good choices about when and how to spend this fund to best create maximal colonization, according to the initially- chosen measure.

That’s the basic idea. Now here’s a variation, designed to avoid incentives for sabotage. When a donor donates $2 to the space fund, $1 goes into the fund, and this donor gets back a $LNYD asset whose value is guaranteed to fall within [$0,$1]. They can then trade this $LNYD asset in the decision markets. The remaining $(1-LNYD) asset is put into in a new space fund tied to a new goal defined regarding some date D’ after date D. In this system, only this new fund holds the $(1-LNYD) assets that might tempt a holder to sabotage the space colonization effort.

GD Star Rating
loading...
Tagged as: ,

Speculator-Chosen Immigrants

On immigration, the big political tug-o-war axis today is: more or less immigrants. But if you want tug the rope sideways, both to oppose polarization and to have a better chance of adding value, you might do better to focus on a perpendicular axis. Such as transferable citizenship, crime liability insurance for immigrants, or the topic of this post: who exactly to admit.

Even if we disagree on how many immigrants we want, we should agree that we want better immigrants. For example, good immigrants pay lots of taxes, volunteer to help their communities, don’t greatly harm our political or social equilibria, are not criminals, and impose fewer burdens on government benefit systems. Yes, we may disagree on the relative weights to assign to such features, but these disagreements seem relatively modest; there’s plenty of room here to work together to make better choices.

Note that, for the foreseeable future, we aren’t likely to approve for immigration more than a small fraction of all the outsiders who’d be willing to apply, if we were likely to accept them. So as a practical matter our efforts to pick candidates should focus on estimating well at the high tail of the distribution, for the candidates most likely to be best.

Note also that while a better way to select immigrants might induce us to accept more immigrants, those who are wary of this outcome tend to feel risk averse about such changes. Thus we should be looking for ways to pick immigrants that seem especially good at assuring skeptics that any one person is a good candidate.

To achieve all this, I suggest that we look at the prices of new financial assets that we can create to track the net tax revenue from each immigrant, conditional on their being admitted. Let me explain.

For every immigrant that we admit, the government could track how much that person pays in taxes each year, and also how much the government spends on that person via benefits whose costs can be measured individually. We could probably assign individual costs for schools, Medicare and Medicaid, prison, etc. For types of costs or benefits that can’t be measured individually, we’d could attribute to each immigrant some average value across citizens of their location and demographic type. When there are doubts, let us err in the direction of estimating higher costs, so that our measures are biased against immigrants adding value.

Okay, so now we have a conservative net financial value number for each immigrant for each year, a number that can be positive or negative. From these numbers we can create financial assets that pay annual dividends proportional to these numbers. If we let many people trade such assets, their market prices should give us decent estimates of the current present financial value of this stream of future revenue. And if we allow trading in such assets regarding people who apply to immigrate, with those trades being conditional on that person being admitted and coming, then such prices would estimate the net financial value of an immigration candidate conditional on their immigrating.

We could then admit the candidates for whom such estimates are highest; using a high threshold could ensure a high confidence that each immigrant is a net financial advantage. Those who are skeptical about particular immigrants, or about immigration in general, could insure themselves against bad immigration choices via trades in these markets, trades from which they expect to profit if their skepticism is accurate.

As usual, there are some subtitles to consider. For example, traders must be given some info on each candidate, and market estimates are more accurate the more info that traders are given. While I see no obvious legal requirement to do so, candidates could be assured some privacy. Immigration skeptics, however, might want to limit such privacy, to better ensure that each immigrant is a net gain.

Once immigrants become citizens, they of course have stronger privacy rights. While the government-calculated dividend values on them each year would reveal some info, there’s no need to reveal details of how that number was computed. To cut info revealed further, we might even wait and pay dividends as a single lump every five years.

In principle, a trader might acquire a large enough net negative stake in a particular immigrant that they have an incentive to hurt that immigrant, or at least to hurt that immigrant’s chances of achieving high net value. We might thus want to limit the size of negative stakes, at least after the immigrant comes, and among traders with opaque abilities to cause such harms.

The fact that net financial revenue can be both positive and negative complicates the asset creation. We might add some large constant to the financial numbers, to ensure that dividends paid have a positive sign. Or we might create two assets, one that pays dividends for the positive amounts, and one that pays for the negative amounts.

Some groups of candidates, such as a church, family, or firm, might be worth more if admitted as a unit together. We might then have trades on packages of assets for a whole group of candidates, trades conditional on their all being admitted as a unit. With a high enough estimated value of the group, we might then just admit such groups as units, even when we have doubts about individual members.

And that’s it, another pull-the-rope-sideways proposal designed to improve policy on a hot-button topic without taking a side on topic’s main dispute. Whether you want more or fewer immigrants, you should want better immigrants.

Added 1p 25Mar: If we could design individual measures of cultural assimilation and impact on cultural change, and assign dollar values to those measures, then we could include them in this proposed system.

GD Star Rating
loading...
Tagged as: , ,

Can We Trust Deliberation Priests?

In Science, academic “deliberation” experts offer a fix for our political ills:

Citizens to express their views … overabundance [of] … has been accompanied by marked decline in civility and argumentative complexity. Uncivil behavior by elites and pathological mass communication reinforce each other. How do we break this vicious cycle? …

All survey research … obtains evidence only about the capacity of the individual in isolation to reason about politics. … [But] even if people are bad solitary reasoners, they can be good group problem-solvers … Deliberative experimentation has generated empirical research that refutes many of the more pessimistic claims about the citizenry’s ability to make sound judgments.

Great huh? But there’s a catch:

Especially when deliberative processes are well-arranged: when they include the provision of balanced information, expert testimony, and oversight by a facilitator … These effects are not necessarily easy to achieve; good deliberation takes time and effort. Many positive effects are demonstrated most easily in face-to-face assemblies and gatherings, which can be expensive and logistically challenging at scale. Careful institutional design involv[es] participant diversity, facilitation, and civility norms …

A major improvement … might involve a randomly selected citizens’ panel deliberating a referendum question and then publicizing its assessments for and against a measure … problem is not social media per se but how it is implemented and organized. Algorithms for ranking sources that recognize that social media is a political sphere and not merely a social one could help. …

It is important to remain vigilant against incentives for governments to use them as symbolic cover for business as usual, or for well-financed lobby groups to subvert their operation and sideline their recommendations. These problems are recognized and in many cases overcome by deliberative practitioners and practice. … The prospects for benign deployment are good to the degree that deliberative scholars and practitioners have established relationships with political leaders and publics—as opposed to being turned to in desperation in a crisis.

So ordinary people are capable of fair and thoughtful deliberation, but only via expensive processes carefully managed in detail by, and designed well in advance by, proper deliberation experts with “established relationships with political leaders and publics.” That is, these experts must be free to pick the “balance” of info, experts, and participants included, and even who speaks when how, and these experts must be treated with proper respect and deference by the public and by political authorities.

No, they aren’t offering a simple well-tested mechanism (e.g., an auction) that we can apply elsewhere with great confidence that the deployed mechanism is the same as the one that they tested. Because what they tested instead was a mechanism with a lot of “knobs” that need context-specific turning; they tested the result of having particular experts use a lot of discretion to make particular political and info choices in particular contexts. They say that went well, and their academic peer reviewers (mostly the same people) agreed. So we shouldn’t worry that such experts would become corrupted if we gave them a lot more power.

This sure sounds like a priesthood to me. If we greatly empower and trust a deliberation priesthood, presumably overseen by these 20 high priest authors and their associates, they promise to create events wherein ordinary people talk much more reasonably, outputting policy recommendations that we could then all defer to with more confidence. At least if we trust them.

In contrast, I’ve been suggesting that we empower and trust prediction markets on key policy outcomes. We’ve tested such mechanisms a lot, including in contexts with strong incentives to corrupt them, and these mechanisms have far fewer knobs that must be set by experts with discretion. Which seems more trustworthy to me.

GD Star Rating
loading...
Tagged as: , , ,

Replication Markets Team Seeks Journal Partners for Replication Trial

An open letter, from myself and a few colleagues:

Recent attempts to systematically replicate samples of published experiments in the social and behavioral sciences have revealed disappointingly low rates of replication. Many parties are discussing a wide range of options to address this problem.

Surveys and prediction markets have been shown to predict, at rates substantially better than random, which experiments will replicate. This suggests a simple strategy by which academic journals could increase the rate at which their published articles replicate. For each relevant submitted article, create a prediction market estimating its chance of replication, and use that estimate as one factor in deciding whether to publish that article.

We the Replication Markets Team seek academic journals to join us in a test of this strategy. We have been selected for an upcoming DARPA program to create prediction markets for several thousand scientific replication experiments, many of which could be based on articles submitted to your journal. Each market would predict the chance of an experiment replicating. Of the already-published experiments in the pool, approximately one in ten will be sampled randomly for replication. (Whether submitted papers could be included in the replication pool depends on other teams in the program.) Our past markets have averaged 70% accuracy, and the work is listed at the Science Prediction Market Project page, and has been published in Science, PNAS, and Royal Society Open Science.

While details are open to negotiation, our initial concept is that your journal would tell potential authors that you are favorably inclined toward experiment article submissions that are posted at our public archive of submitted articles. By posting their article, authors declare that they have submitted their article to some participating journal, though they need not say which one. You tell us when you get a qualifying submission, we quickly tell you the estimated chance of replication, and later you tell us of your final publication decision.

At this point in time we seek only an expression of substantial interest that we can take to DARPA and other teams. Details that may later be negotiated include what exactly counts as a replication, whether archived papers reveal author names, how fast we respond with our replication estimates, what fraction of your articles we actually attempt to replicate, and whether you privately give us any other quality indicators obtained in your reviews to assist in our statistical analysis.

Please RSVP to: Angela Cochran, PM acochran@replicationmarkets.com 571 225 1450

Sincerely, the Replication Markets Team

Thomas Pfeiffer (Massey University)
Yiling Chen, Yang Liu, and Haifeng Xu (Harvard University)
Anna Dreber Almenberg & Magnus Johannesson (Stockholm School of Economics)
Robin Hanson & Kathryn Laskey (George Mason University)

Added 2p: We plan to forecast ~8,000 replications over 3 years, ~2,000 within the first 15 months.  Of these, ~5-10% will be selected for an actual replication attempt.

GD Star Rating
loading...
Tagged as: , ,

Toward An Honest Consensus

Star Trek original series featured a smart computer that mostly only answered questions; humans made key decisions. Near the start of Nick Chater’s book The Mind Is Flat, which I recently started, he said early AI visions were based on the idea of asking humans questions, and then coding their answers into a computer, which might then answer the same range of questions when asked. But to the surprise of most, typical human beliefs turned out to be much too unstable, unreliable, incoherent, and just plain absent to make this work. So AI research turned to other approaches.

Which makes sense. But I’m still inspired by that ancient vision of an explicit accessible shared repository of what we all know, even if that isn’t based on AI. This is the vision that to varying degrees inspired encyclopedias, libraries, internet search engines, prediction markets, and now, virtual assistants. How can we all coordinate to create and update an accessible shared consensus on important topics?

Yes, today our world contains many social institutions that, while serving other functions, also function to create and update a shared consensus. While we don’t all agree with such consensus, it is available as a decent first estimate for those who do not specialize in a topic, facilitating an intellectual division of labor.

For example: search engines, academia, news media, encyclopedias, courts/agencies, consultants, speculative markets, and polls/elections. In many of these institutions, one can ask questions, find closest existing answers, induce the creation of new answers, induce elaboration or updates of older answers, induce resolution of apparent inconsistencies between existing answers, and challenge existing answers with proposed replacements. Allowed questions often include meta questions such as origins of, translations of, confidence in, and expected future changes in, other questions.

These existing institutions, however, often seem weak and haphazard. They often offer poor and biased incentives, use different methods for rather similar topics, leave a lot of huge holes where no decent consensus is offered, and tolerate many inconsistencies in the answers provided by different parts. Which raises the obvious question: can we understand the advantages and disadvantages of existing methods in different contexts well enough to suggest which ones we should use more or less where, or to design better variations, ones that offer stronger incentives, lower costs, and wider scope and integration?

Of course computers could contribute to such new institutions, but they needn’t be the only or even main parts. And of course the idea here is to come up with design candidates to test first at small scales, scaling up only when results look promising. Design candidates will seem more promising if we can at least imagine using them more widely, and if they are based on theories that plausibly explain failings of existing institutions. And of course I’m not talking about pressuring people to follow a consensus, just to make a consensus available to those who want to use it.

As usual, a design proposal should roughly describe what acts each participant can do when, what they each know about what others have done, and what payoffs they each get for the main possible outcomes of typical actions. All in a way that is physically, computationally, and financially feasible. Of course we’d like a story about why equilibria of such a system are likely to produce accurate answers fast and at low cost, relative to other possible systems. And we may need to also satisfy hidden motives, the unacknowledged reasons for why people actually like existing institutions.

I have lots of ideas for proposals I’d like the world to consider here. But I realized that perhaps I’ve neglected calling attention to the problem itself. So I’ve written this post in the hope of inspiring some of you with a challenge: can you help design (or test) new robust ways to create and update a social consensus?

GD Star Rating
loading...
Tagged as: , ,

Choose: Allies or Accuracy

Imagine that person A tells you something flattering or unflattering about person B. All else equal, this should move your opinion of B in the direction of A’s claim. But how far? If you care mainly about accuracy, you’ll want to take into account base rates on claimers A and targets B, as well as more specific specific signs on the accuracy of A regarding B.

But what if you care mainly about seeming loyal to your allies? Well if A is more of your ally than is B, as suggested by your listening now to A, then you’ll be more inclined to just believe A, no matter what. Perhaps if other allies give a different opinion, you’ll have to decide which of your allies to back. But if not, trying to be accurate on B mainly risks seeming disloyal to A and you’re other allies.

It seems that humans tend to just believe gossip like this, mostly ignoring signs of accuracy:

The trustworthiness of person-related information … can vary considerably, as in the case of gossip, rumors, lies, or “fake news.” …. Social–emotional information about the (im)moral behavior of previously unknown persons was verbally presented as trustworthy fact (e.g., “He bullied his apprentice”) or marked as untrustworthy gossip (by adding, e.g., allegedly), using verbal qualifiers that are frequently used in conversations, news, and social media to indicate the questionable trustworthiness of the information and as a precaution against wrong accusations. In Experiment 1, spontaneous likability, deliberate person judgments, and electrophysiological measures of emotional person evaluation were strongly influenced by negative information yet remarkably unaffected by the trustworthiness of the information. Experiment 2 replicated these findings and extended them to positive information. Our findings demonstrate a tendency for strong emotional evaluations and person judgments even when they are knowingly based on unclear evidence. (more; HT Rolf Degen)

I’ve toyed with the idea of independent juries to deal with Twitter mobs. Pay a random jury a modest amount to 1) read a fuller context and background on the participants, 2) talk a bit among themselves, and then 3) choose which side they declare as more reasonable. Sure sometimes the jury would hang, but often they could give a voice of reason that might otherwise be drown out by loud participants. I’d have been willing to pay for this a few times. And once juries became a standard thing, we could lower costs via making prediction markets on jury verdicts if a case were randomly choose for jury evaluation.

But alas, I’m skeptical that most would care much about what an independent jury is estimated to say, or even about what it actually says. For that, they’d have to care more about truth than about showing support for allies.

GD Star Rating
loading...
Tagged as: , ,