Shoulda-Listened Futures

Over the decades I have written many times on how prediction markets might help the intellectual world. But usually my pitch has been to those who want to get a better actionable info out of intellectuals, or to help the world to make better intellectual progress in the long run. Problem is, such customers seem pretty scarce. So in this post I want to outline an idea that is a bit closer to a business proposal, in that I can better identify concrete customers who might pay for it.

For every successful intellectual there are (at least) hundreds of failures. People who started out along a path, but then were not sufficiently rewarded or encouraged, and so then either quit or persisted in relative obscurity. And a great many of these (maybe even a majority) think that the world done them wrong, that their intellectual contributions were underrated. And no doubt many of them are right. Such malcontents are my intended customers.

These “world shoulda listened to me” customers might pay to have some of their works evaluated by posterity. For example, for every $1 saved now that gains a 3% real rate of return, $19 in real assets are available in a century to pay historians for evaluations. At a 6% rate of return (or 3% for 2 centuries), that’s $339. Furthermore, if future historians needed only to randomly evaluate 1% of the works assigned them, then if malcontents paid $10 per work to be maybe evaluated, historians could spend $20K (or $339K) per work they evaluate. Considering all the added knowledge and tools to which future historians may have access, that seems enough to do a substantial evaluation, especially if they evaluate several related works at the same time.

Given a substantial chance (1% will do) that a work might be evaluated by historians in a century or two, we could then create (conditional) prediction markets now estimating those future evaluations. So a customer might pay their $20 now, and get an immediate prediction market estimate of that future evaluation for their work. That $20 might pay $10 for the (chance of a) future evaluation and another $10 to establish and subsidize a prediction market over the coming centuries until resolution.

Finally, if customers thought market estimate regarding their works looked too low, then they could of course try to bet to raise those estimates. Skeptics would no doubt lie waiting to bet against them, and on average this tendency of authors to bet to support their works would probably subsidize these markets, and so lower the fees that the system needs to charge.

Of course even with big budgets for evaluations, if we want future historians to make reliable enough formal estimates that we can bet on in advance, then we will need to give them a well-defined-enough task to accomplish. And we need to define this task in a way that discourages future historians from expressing their gratitude to all these people who funded their work by giving them all an A+.

I suggest we have future historians estimate each work’s ideal attention: how much attention each particular work should have been given during some time period. So we should pick some measure of attention, a measure that we can calculate for works when they are submitted, and track over time. This measure should weigh if the dissertation was approved, the paper was published and where, how many cites did it get, etc. If we add up all the initial attention for submitted works, then we can assign historians the task of (counterfactually) reallocating this total attention across all the submitted works. So to give more attention to some, they’d have to take away attention from others.

Okay, so now they can’t give every work an A+. (And we ensure that bet assets have bounded values.) But our job isn’t done. We also need to give them a principle to follow when allocating attention among all these prior works. What objective would they be trying to accomplish via this reallocation of attention?

I suggest that the objective just be intellectual progress, toward the world having access to more accurate and useful beliefs. A set of works should have gotten more attention if in that case the world would have been more likely to have more quickly come to appreciate valuable truths. And this task is probably easier if we ask future historians to use their future values in this task, instead of asking them to try to judge according to our values today.

These evaluation tasks probably get easier if historians randomly pick related sets of works to evaluate together, instead of independently picking each work to evaluate. And this system can probably offer scaled fees, wherein the chance that your work gets evaluated rises linearly with the price you paid for that chance. There are probably a lot more details to work out, but I expect I’ve already said enough for most people to decide roughly how much they like this idea.

Once there were many works in this system, and many prediction markets estimating their shoulda-been attention, then we could look to see if market speculators see any overall biases in today’s intellectual worlds. That is, topics, methods, disciplines, genders, etc. to which speculators estimate that the world today is giving too little attention. That could be pretty dramatic and damning evidence of bias, by someone, evidence to which we’d all be wise to attend.

One obvious test of this approach would be to assign historians today the task of reallocating attention among papers published a century or two ago. Perhaps assign multiple independent groups, and see how correlated are their evaluations, and how that correlation varies across topic areas. Perhaps repeating in a decade or two, to see how much evaluations drift over time.

Showing these correlations to potential customers might convince them that there’s a good enough chance that such a system will later correctly vindicate their neglected contributions. And these tests may show good scopes to use, for related works and time periods to evaluate together, and how narrow or broad should be the expertise of the evaluators.

This whole shoulda-listened-futures approach could or course also be applied to many other kinds of works, not just intellectual works. You’d just have to establish your standards for how future historians are to allocate shoulda attention, and trust them to actually follow those standards. Doing tests on works from centuries ago here could also help to show if this is a viable approach for these kinds of works.

Added 7am 28Apr: On average more assets will be available to pay for future evaluations if the fees paid are invested in risky assets. So instead of promising a particular percentage chance of evaluation, it may make more sense to specify how fees will be invested, set the (real) amount to be spent on each evaluation, and then promise that the chance of evaluation for each work will be set by the investment return relative to the initial fee paid. Yes that induces more evaluations in state of the world where investments do better, but customers are already accepting a big chance that their work will never be directly evaluated.