An interesting paradox arise if one tries to apply Aumann’s theorem in a world of widespread disagreement. The problem is especially apparent in considering the stronger principle which Eliezer calls the Modesty Argument, which claims not only that rational, mutually-respecting people should not agree to disagree, but that in general one should be agreeable and allow oneself to be easily persuaded in any argument.

I’ll introduce it with one of my favorite jokes:

This is a story about advice, and any story about advice becomes a story about a village rabbi. Two men came to see the rabbi of their village.

The first one said, "Rabbi, I have a pear tree in my yard. My father planted it, I keep it watered and don’t let the chickens peck on its roots. One of its branches hangs over my neighbor’s wall. So what do I see yesterday but my neighbor standing there eating one of my pears. This is theft, and I want him to pay me damages."

The rabbi nodded his head. "You’re right, you’re right."

The other neighbor said, "Rabbi, you know that I have seven children–without my garden to feed them, how would I manage? But that tree casts a shadow where nothing will grow. So yesterday, when I go out to dig some potatoes, a pear from his tree falls and hits me right on the head. How am I hurting my neighbor if I eat it? And doesn’t he owe me something for his tree’s blocking sunlight?

The rabbi thought and then said, "You’re right, you’re right."

Meanwhile, the rabbi’s wife had been hearing all this. "How can you say, ‘You’re right’ to both of these men? Surely one of the men is right, and the other is wrong!"

The rabbi looked unhappy. "You’re right, you’re right."

Suppose that I am somewhat neutral on a given issue and I am going to argue with one of two partisans on the issue, who each have opposing views. Applying the Modesty Argument it might seem that I should be prepared to be persuaded to the view of whichever one of them I interact with. It is true, in talking to one of them, that we both know that other people exist with opposing views, but since this person has not been persuaded by this knowledge, he must have good reason to hold to his particular side. There is no reason a priori to suppose that my beliefs on this are any better founded than his, especially since we stipulated that it was a position on which I was relatively ignorant. So it is plausible, given that we reach agreement, and given his long-standing commitment to his position, that our resulting agreement will generally be favorable to his side of the argument.

The problem is that I can equally well reason that if I were to interact instead with the person who takes the opposite view, I would be just as convinced to take that other side. This evokes a rather comical image of sequential interactions with the two partisans, in which I am first convinced of one side and then the other. It seems that someone following the Modesty Argument too literally will find himself in the position of the poor rabbi in the joke.

The problem and paradox is that this behavior is irrational. You can’t be in a position where you have an expectation that you can gain some information that will move your opinion in a predictable direction. Just knowing this fact about the situation should be enough to move your opinion already. Robin has what I think is a formal proof of this in his paper, Disagreement is Unpredictable (not sure I understood it correctly though). For a rational person, every new piece of information is a surprise in the sense that he can’t predict which way it will change his opinion, on the average. When I post these messages here I have no idea whether I will be more or less convinced of my opinion as a result of the feedback I receive. That is how it always must be.

Therefore I can’t be in a position where I know if I talk to person A I will be convinced that he is right, and if I talk to person B I will be convinced that A is wrong. A practitioner of the Modesty Argument who finds himself in this situation is doing something wrong.

So there are two questions: what is wrong with the argument above that puts us in the shoes of the rabbi; and what in fact should a rational and modest person do in a world full of disagreement?

First, I seem to have gone wrong when I concluded that if I argued with person A, I would come to agree with him. As I just pointed out, it is irrational to have an expectation that gaining information will predictably change one’s opinion. Instead, it must be the case that I am completely uncertain about how my opinion will change if I talk with A. Given that we will agree, it must be my expectation that A is as likely to change his mind as that he will convince me. Now, I argued above that this is unlikely given A’s long-standing position as a partisan in the dispute. Therefore I have to conclude that A does not meet the conditions for Aumann’s result; he is not rational, or honest, or at least he assumes that those he disagrees with lack these traits. In that case I cannot apply Aumann’s theorem and cannot act in accordance with the Modesty Argument in my discussions with A or B.

It’s interesting to consider what would happen if A and B are in fact both rational and honest, but both distrust whether their counterpart shares these properties. This is a stable configuration, and it’s rational for A and B to disagree in this situation; but of course they lack the mutual respect which is generally associated with saying that they agree to disagree. Now suppose A and B both trust me as an "honest broker", and I trust them. Then I can’t disagree with A, and I can’t disagree with B, once I interact with them. What will happen is that when I interact with A, we two will agree; and when I interact with B, we two will agree. In this way, A and B will come to agreement by both agreeing with me (I might have to repeat the interaction a few times). This again illustrates how odd is the persistence of disagreement in the world, among people who claim to be rational and honest.

But realistically, disagreement does persist, which leaves the question of how a rational person should behave. Unfortunately, I don’t think that Aumann’s theorem or the Modesty Argument offers much guidance. It’s not rational to adopt a stance where you are easily persuaded by everyone you talk to. There’s no particular reason, just because you are talking to a proponent of one side, to believe him, given that you know there are plenty of people around who believe the opposite. It’s not that you think you are better (smarter, more knowledgeable, more rational) than he is; it’s just that you assume there are many equally smart, knowledgeable and rational people on the other side. In the end, you have to make your decision on controversial issues using other grounds than the irrationality of disagreement.