This week Nature published some empirical data on a surprising-popularity consensus mechanism (a previously published mechanism, e.g., Science in 2004, with variations going by the name “Bayesian Truth Serum”). The idea is to ask people to pick from several options, and also to have each person forecast the distribution of opinion among others. The options that are picked surprisingly often, compared to what participants on average expected, are suggested as more likely true, and those who pick such options as better informed.
I'd like to repost a highly voted comment over from LW about an interesting failure mode of the SP heuristic:
> whpearson 03 February 2017 08:35:51PM 9 points
> I wonder how well it would work on questions like.
> "Does homeopathy cure cancer".
> Or in general where there are people in the minority that know the majority won't side with them, but the majority might not know how many believe the fringe view.
The morality of abortion doesn't reduce to "widely shared moral criteria" for most ostensible experts. Moral realism is alive and well, however misguided. As I understand him, the founder of Less Wrong is a moral realist, for whom morality doesn't reduce to "widely shared moral criteria."
But MY point is that the first-order strategy is still for the neural net to simply answer the object-level question to the beset of its ability, and if you think that there is some second-order strategy, it's incumbent on you to explain why. Even if the copy isn't perfect, and I know that its answer will vary from mine, unless I have some reason to expect it to vary in particular direction, my best strategy is to simply give whatever answer I think is correct, and figure that any variances among copies of me are clustered around that answer (or, more precisely, all copies of me employing that strategy will cluster around that being the correct strategy).
True, even if we ran a long-term controlled experiment it wouldn’t be clear as to what we would be looking for in the resulting data, how to decide which system would have outperformed the other.
If only a few people knew the answer and everyone else answered randomly, then yes the mechanism would work. But if many people answer randomly and then a large group coordinates to an incorrect answer and a small group coordinates to a correct answer, this mechanism will give and reward the first incorrect answer.
>Of course even in this case your incentive is to report the city that most sources would say is the capital. If you in fact know that according to the detailed legal history another city is rightfully the capital, not the city that the usual records give, your incentive is still to go with usual records.
I believe that's incorrect. In fact, from the Nature paper:
> Imagine that there are two possible worlds, the actual one in which Philadelphia is not the capital of Pennsylvania, and the counterfactual one in which Philadelphia is the capital. It is plausible that in the actual world fewer people will vote yes than in the counterfactual world. This can be formalized by the toss of a biased coin where, say, the coin comes up yes 60% of the time in the actual world and 90% of the time in the counterfactual world. Majority opinion favours yes in both worlds. People know these coin biases but they do not know which world is actual. Consequently, their predicted frequency of yes votes will be between 60% and 90%. However, the actual frequency of yes votes will converge to 60% and no will be the surprisingly popular, and correct, answer.
So if you're the only one that knows that a certain city is the true capital, then the method doesn't work; but if there a reasonable minority that knows that, then the method works.
"This mechanism can thus be applied to most any topic, such as the morality of abortion, the existence of God, or the location of space aliens."
Nonsense. The first one is not a factual question, and the second is not well formed, not verifiable, is in a completely different reference class, and has only two main answers. Only the last comes close to being a case where this is applicable.
If you tell me that you've created a perfect copy of me in a simulation, and ask me to predict what what it will say is the capital of Nebraska, my best strategy is to simply say whatever I think the capital of Nebraska is. How will training neural network to predict other neural networks be different?
Why the mistrial part? If the independent votes do not tally to the required quorum, the defendant is not guilty. The required quorum might need to be amended though.
I think that's still wrong. The majority wrong answer will have the most answers, but it won't be surprisingly popular.
The minority answer will be surprisingly popular, since the majority won't consider it a likely answer.
I'd like to repost a highly voted comment over from LW about an interesting failure mode of the SP heuristic:
> whpearson 03 February 2017 08:35:51PM 9 points
> I wonder how well it would work on questions like.
> "Does homeopathy cure cancer".
> Or in general where there are people in the minority that know the majority won't side with them, but the majority might not know how many believe the fringe view.
To which I say: of course not.
The morality of abortion doesn't reduce to "widely shared moral criteria" for most ostensible experts. Moral realism is alive and well, however misguided. As I understand him, the founder of Less Wrong is a moral realist, for whom morality doesn't reduce to "widely shared moral criteria."
But MY point is that the first-order strategy is still for the neural net to simply answer the object-level question to the beset of its ability, and if you think that there is some second-order strategy, it's incumbent on you to explain why. Even if the copy isn't perfect, and I know that its answer will vary from mine, unless I have some reason to expect it to vary in particular direction, my best strategy is to simply give whatever answer I think is correct, and figure that any variances among copies of me are clustered around that answer (or, more precisely, all copies of me employing that strategy will cluster around that being the correct strategy).
True, even if we ran a long-term controlled experiment it wouldn’t be clear as to what we would be looking for in the resulting data, how to decide which system would have outperformed the other.
Of course.
The point is you can't evaluate the value of deliberation other than with respect to the other factors, and what your goals are.
If only a few people knew the answer and everyone else answered randomly, then yes the mechanism would work. But if many people answer randomly and then a large group coordinates to an incorrect answer and a small group coordinates to a correct answer, this mechanism will give and reward the first incorrect answer.
>This mechanism can thus be applied to most any topic, such as the morality of abortion
That question will reduce to something like: "according to widely shared moral criteria, is abortion moral?"
http://lesswrong.com/r/disc...
The whole point is that the copy is imperfect.
>Of course even in this case your incentive is to report the city that most sources would say is the capital. If you in fact know that according to the detailed legal history another city is rightfully the capital, not the city that the usual records give, your incentive is still to go with usual records.
I believe that's incorrect. In fact, from the Nature paper:
> Imagine that there are two possible worlds, the actual one in which Philadelphia is not the capital of Pennsylvania, and the counterfactual one in which Philadelphia is the capital. It is plausible that in the actual world fewer people will vote yes than in the counterfactual world. This can be formalized by the toss of a biased coin where, say, the coin comes up yes 60% of the time in the actual world and 90% of the time in the counterfactual world. Majority opinion favours yes in both worlds. People know these coin biases but they do not know which world is actual. Consequently, their predicted frequency of yes votes will be between 60% and 90%. However, the actual frequency of yes votes will converge to 60% and no will be the surprisingly popular, and correct, answer.
So if you're the only one that knows that a certain city is the true capital, then the method doesn't work; but if there a reasonable minority that knows that, then the method works.
The first one is not a factual question...
It is for a moral realist, and in another posting, Robin reports that the "expert concensus" supports moral realism.
[Of course, belief in moral realism is actually as absurd as belief in deities.]
"This mechanism can thus be applied to most any topic, such as the morality of abortion, the existence of God, or the location of space aliens."
Nonsense. The first one is not a factual question, and the second is not well formed, not verifiable, is in a completely different reference class, and has only two main answers. Only the last comes close to being a case where this is applicable.
If you tell me that you've created a perfect copy of me in a simulation, and ask me to predict what what it will say is the capital of Nebraska, my best strategy is to simply say whatever I think the capital of Nebraska is. How will training neural network to predict other neural networks be different?
Why the mistrial part? If the independent votes do not tally to the required quorum, the defendant is not guilty. The required quorum might need to be amended though.
Compared to what? Have each juror make a separate vote, and if the first vote is inconclusive, have a mistrial?