Imagine a world of people doing various specific projects, where over the long run the net effect of all these projects is to produce some desired outcomes. These projects may interact in complex ways. To encourage people to do more and better such projects along the way, we might like a way to eventually allocate credit to these various projects for their contributions to desired outcomes.
1) US presidents don't have as much power over the US as people think they do.2) When presidents can make changes, the impact of those changes is only felt a couple of years after the fact.
Given 1 and 2, it isn't clear to me that overseeing the "rock-bottom worst periods" of US history is an indictment.
Historian rankings are notoriously bad. Historian rankings of US presidents consistently rank the two presidents who oversaw the two rock-bottom worst periods of US history in the top 3, sometimes as #1 and #2. Why would we trust these guys?
So you mentioned legal judges, peer reviewers, and current historians.
The easiest one to answer is current historians. Current historians are more reliable than your proposed historian judges, because there is no strong financial interest pressuring current historians to judge one way or the other. For the most part, current historians are academics who are evaluated by their peers, and which outside influences largely don't care about. Your proposed future historian judges would be under a lot more external pressure to vote one way or the other. See Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."
Even so, current historians are subject to a great deal of cultural bias. US History as taught in the 1950s is quite a bit different from US History as taught today. Many historians of today would consider historians of the 1950s as biased - over-emphasizing the role of white men, for example. And historians of the 1950s would likely be similarly displeased with many historians of today. Whoever you think is right, there's clearly a very large cultural influence to at least one group's judgments.
Current peer reviewers are more reliable than your proposed historian judges because, first, the process is at least in theory anonymized, with reviewers supposedly not knowing who the submitter is and vice versa. Second, there aren't usually huge external financial influences for the reviewer to decide one way or the other, and the anonymization helps prevent this too. Third, the scope of a reviewer's judgments is (supposed to be) limited to pointing out specific flaws in the paper's methods, reasoning, and formatting, rather than the reviewer just saying whether they agree with the conclusion. This makes the reviewer's tasks more specific and verifiable than the task of your historian judges (which is to broadly and subjectively estimate credit).
Legal judges again operate on specific and verifiable questions of law. Most of a judge's work involves applying existing law according to precedent, rather than setting new precedent. We don't give judges complete freedom to decide who they think is most responsible for a crime, the way your proposed historian judges are deciding credit; legal judges apply specific rules with only partial leeway.
Even so, current legal judges are often subject to powerful outside financial influences. I don't think it's necessary to go into details, but the law is very often not fair to the little guy. Powerful corporations are much more likely to get a slap on the wrist for large-scale injustices, while minor crimes send poor people to jail for decades.
Legal courts today use juries and judges to decide cases. Academia today uses supposed independent judges in peer review. Historians today actually do express judgements on history. Why would my proposal work worse than these do now?
We might also make sure a lot of historian pay for this work is based on the judgment of further future historians.
The challenge, of course, is finding a fair judging method. We would expect judges to be strongly biased by the political climate at the time of judging. Would you trust PRC judges to decide the credit for scientific achievements, some of which were Chinese and some of which were not? Or in general any historian is likely to give more credit to projects within his country or educational institution.
Of course, any investors in the complex impact futures would have a very strong incentive to bribe or otherwise influence the judges. Apart from direct bribes, the investors could give grants to the universities training the judges, on condition that the institutions hire/fire certain instructors, or fund the study of the subject on which they hold the future. The investors could also lobby for national K-12 education standards or TV specials that emphasize certain events and de-emphasize others, which would influence future historians while they are children.
Like other people, historians have a bias towards thinking that what is focal is causal. Even without obviously pushing any point of view, simply devoting more research time and funding to a historical subject will lead the historians to give the subject more credit for future events.
The most fundamental problem is that there is no ultimate reality check on the historians' judgments. What they say decides the market - end of story - with no opportunity for them to be proved wrong by hard evidence.
This is a problem that can't be solved with funding alone. It may be solvable, but it's going to take some good ideas.
1) US presidents don't have as much power over the US as people think they do.2) When presidents can make changes, the impact of those changes is only felt a couple of years after the fact.
Given 1 and 2, it isn't clear to me that overseeing the "rock-bottom worst periods" of US history is an indictment.
Historian rankings are notoriously bad. Historian rankings of US presidents consistently rank the two presidents who oversaw the two rock-bottom worst periods of US history in the top 3, sometimes as #1 and #2. Why would we trust these guys?
So you mentioned legal judges, peer reviewers, and current historians.
The easiest one to answer is current historians. Current historians are more reliable than your proposed historian judges, because there is no strong financial interest pressuring current historians to judge one way or the other. For the most part, current historians are academics who are evaluated by their peers, and which outside influences largely don't care about. Your proposed future historian judges would be under a lot more external pressure to vote one way or the other. See Goodhart's Law: "When a measure becomes a target, it ceases to be a good measure."
Even so, current historians are subject to a great deal of cultural bias. US History as taught in the 1950s is quite a bit different from US History as taught today. Many historians of today would consider historians of the 1950s as biased - over-emphasizing the role of white men, for example. And historians of the 1950s would likely be similarly displeased with many historians of today. Whoever you think is right, there's clearly a very large cultural influence to at least one group's judgments.
Current peer reviewers are more reliable than your proposed historian judges because, first, the process is at least in theory anonymized, with reviewers supposedly not knowing who the submitter is and vice versa. Second, there aren't usually huge external financial influences for the reviewer to decide one way or the other, and the anonymization helps prevent this too. Third, the scope of a reviewer's judgments is (supposed to be) limited to pointing out specific flaws in the paper's methods, reasoning, and formatting, rather than the reviewer just saying whether they agree with the conclusion. This makes the reviewer's tasks more specific and verifiable than the task of your historian judges (which is to broadly and subjectively estimate credit).
Legal judges again operate on specific and verifiable questions of law. Most of a judge's work involves applying existing law according to precedent, rather than setting new precedent. We don't give judges complete freedom to decide who they think is most responsible for a crime, the way your proposed historian judges are deciding credit; legal judges apply specific rules with only partial leeway.
Even so, current legal judges are often subject to powerful outside financial influences. I don't think it's necessary to go into details, but the law is very often not fair to the little guy. Powerful corporations are much more likely to get a slap on the wrist for large-scale injustices, while minor crimes send poor people to jail for decades.
Legal courts today use juries and judges to decide cases. Academia today uses supposed independent judges in peer review. Historians today actually do express judgements on history. Why would my proposal work worse than these do now?
We might also make sure a lot of historian pay for this work is based on the judgment of further future historians.
It's an idea that shows promise.
The challenge, of course, is finding a fair judging method. We would expect judges to be strongly biased by the political climate at the time of judging. Would you trust PRC judges to decide the credit for scientific achievements, some of which were Chinese and some of which were not? Or in general any historian is likely to give more credit to projects within his country or educational institution.
Of course, any investors in the complex impact futures would have a very strong incentive to bribe or otherwise influence the judges. Apart from direct bribes, the investors could give grants to the universities training the judges, on condition that the institutions hire/fire certain instructors, or fund the study of the subject on which they hold the future. The investors could also lobby for national K-12 education standards or TV specials that emphasize certain events and de-emphasize others, which would influence future historians while they are children.
Like other people, historians have a bias towards thinking that what is focal is causal. Even without obviously pushing any point of view, simply devoting more research time and funding to a historical subject will lead the historians to give the subject more credit for future events.
The most fundamental problem is that there is no ultimate reality check on the historians' judgments. What they say decides the market - end of story - with no opportunity for them to be proved wrong by hard evidence.
This is a problem that can't be solved with funding alone. It may be solvable, but it's going to take some good ideas.