Discussion about this post

User's avatar
Tim Tyler's avatar

Ideal rational agents with common priors should never have common knowledge of disagreement.

As I pointed out further up the page, such agents must also have truth-seeking as their top priority for this to hold. If they have other goals, they can easily find themselves with irreconcilable differences.

You can surely be rational and not have truth seeking as your primary goal. Rationality and goals are totally orthogonal things - at least in my book. Does the repeated occurrence of this curious idea mean that people are mixing these concepts together?

The fact that humans persistently have common knowledge of disagreements indicates that something is very wrong.

It indicates that humans do not have truth-seeking as their primary goal. Of course, evolutionary theory suggests that agents with truth-seeking as their primary goal can be expected to be rare - so this hardly seems like news to me.

Expand full comment
Overcoming Bias Commenter's avatar

I've written a number of posts on disagreement myself, but I think that most reasonable parties who've been keeping track of the debate, at this point, should confess the following:

1) Ideal rational agents with common priors should never have common knowledge of disagreement.

2) In the real world, two sane rationalists with common knowledge of each other's sanity should not have common knowledge of disagreement. ("Sane" here is a variable that ranges over different definitions of sanity, but it excludes e.g. priors too crazy to reflect on their own causal origins.)

3) The fact that humans persistently have common knowledge of disagreements indicates that something is very wrong.

4) (3) shows that humans systematically overestimate their own meta-rationality, that is, ability to judge whether others are more or less rational than themselves.

5) ...and that, in a lot of cases, Disagreements Aren't About Belief.

It's where we start talking about practical remedies for this dreadful, dreadful situation, that I think we begin entering into the area of - ahem - reasonable disagreement. I don't think the debate has settled the question of what to do when you find yourself disagreeing.

Expand full comment
23 more comments...

No posts