Comments on: Bounded rationality and the conjunction fallacy http://www.overcomingbias.com/2007/09/bounded-rationa.html Overcoming Bias is economist Robin Hanson’s blog, on honesty, signaling, disagreement, forecasting, and the far future. Sat, 11 Feb 2012 23:23:58 +0000 hourly 1 http://wordpress.org/?v=3.1.3 By: Eliezer Yudkowsky http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415175 Eliezer Yudkowsky Sat, 22 Sep 2007 22:44:01 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415175 Douglas, if you have a probability distribution over "impossible possible worlds" and treat the presentation of the proof as evidence causing an update, then it is possible a bounded rationalist could avoid the conjunction fallacy in that sense. The very question itself would act as a kind of evidence about logical truths. Douglas, if you have a probability distribution over “impossible possible worlds” and treat the presentation of the proof as evidence causing an update, then it is possible a bounded rationalist could avoid the conjunction fallacy in that sense. The very question itself would act as a kind of evidence about logical truths.

]]> By: joe http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415174 joe Sat, 22 Sep 2007 22:17:24 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415174 D. Knight, I believe we just cross-posted, but I believe that my post is a more explicit example of what could be happening. Essentially, I am trying to argue that the presentation of A and B should increase your perception of the probability of B if in your previous calculation, you had not considered event A as a possible subset of all events leading to the occurrence of B. D. Knight, I believe we just cross-posted, but I believe that my post is a more explicit example of what could be happening. Essentially, I am trying to argue that the presentation of A and B should increase your perception of the probability of B if in your previous calculation, you had not considered event A as a possible subset of all events leading to the occurrence of B.

]]> By: joe http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415173 joe Sat, 22 Sep 2007 22:00:48 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415173 Actually, if there exists a set of mutually exclusive events, A1,A2, ... An which could result in event A happening, then P(A)= P(A|A1)P(A1) + P(A|A2)P(A2) + ... +P(A|An)P(An) Now suppose you are asked for the probability of A, and you can only think of a subset of A1, A2, ... An, say A1, A2 such that your estimate of P(A)=P(A|A1)P(A1) + P(A|A2)P(A2) is much less than the true probability P(A). Now suppose you are asked for the probability of A and A4. It is possible that your estimate of P(A and A4)=P(A|A4)P(A4) could be much larger than your previous estimate of P(A) which equaled P(A|A1)P(A1) + P(A|A2)P(A2). Clearly, this is a case in which your estimate of P(A) could actually be much smaller than your estimate of P(A and A4), but all this implies is that your original estimate of P(A) was much smaller than it should have been since you didn't include event A4 in your calculation. In this situation, I would not say that your reasoning was fallacious, unless you do not realize that your original estimate of P(A) should have been higher, which seems to be what happens when you are presented the events A as well as A and B, at the same time, yet fallaciously decide that P(A and B) is higher than P(A). Actually, if there exists a set of mutually exclusive events, A1,A2, … An which could result in event A happening, then

P(A)= P(A|A1)P(A1) + P(A|A2)P(A2) + … +P(A|An)P(An)

Now suppose you are asked for the probability of A, and you can only think of a subset of A1, A2, … An, say A1, A2 such that your estimate of
P(A)=P(A|A1)P(A1) + P(A|A2)P(A2)
is much less than the true probability P(A).

Now suppose you are asked for the probability of A and A4. It is possible that your estimate of P(A and A4)=P(A|A4)P(A4) could be much larger than your previous estimate of P(A) which equaled P(A|A1)P(A1) + P(A|A2)P(A2).

Clearly, this is a case in which your estimate of P(A) could actually be much smaller than your estimate of P(A and A4), but all this implies is that your original estimate of P(A) was much smaller than it should have been since you didn’t include event A4 in your calculation. In this situation, I would not say that your reasoning was fallacious, unless you do not realize that your original estimate of P(A) should have been higher, which seems to be what happens when you are presented the events A as well as A and B, at the same time, yet fallaciously decide that P(A and B) is higher than P(A).

]]> By: Douglas Knight http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415172 Douglas Knight Sat, 22 Sep 2007 21:49:17 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415172 <em>But it's still a fallacy. You can't really have P(A&B) > P(B), said the subjective Bayesian.</em> You don't have that in this example. What you have is that the presentation of A&B causes the probability of B to rise. (Unlike in the Linda example, where people claim explicitly that P(A&B) > P(B).) Unless by "subjective Bayesian" you mean that you insist that we treat people as look-up tables of probabilities. I think that point of view has a lot worse problems; eg, the failure of the look-ups to commute is the problem here, not the conjunction fallacy. But it’s still a fallacy. You can’t really have P(A&B) > P(B), said the subjective Bayesian.

You don’t have that in this example. What you have is that the presentation of A&B causes the probability of B to rise. (Unlike in the Linda example, where people claim explicitly that P(A&B) > P(B).)

Unless by “subjective Bayesian” you mean that you insist that we treat people as look-up tables of probabilities. I think that point of view has a lot worse problems; eg, the failure of the look-ups to commute is the problem here, not the conjunction fallacy.

]]> By: joe http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415171 joe Sat, 22 Sep 2007 20:58:33 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415171 "In fact I think that a commenter on an earlier post got it pretty much right when implying that we are implicitly calculating the probability of the story we hear given the claim rather than the probability of the claim given the story, and that this is why we get it wrong." In my 9/20/07 11:47 P.M. post on Burdensome details, I claimed that instead of calculating P(A and B), where B is supporting evidence of how A could happen, they are mistakenly calculating P(A given B) without downweighting the calcualtion correctly by the probability of B. Through rearranging the definition of conditional probability, we find that P(A and B)= P(A given B)*P(B). It could be really easy to either forget, or to not downweight the calculation enough when P(A given B) is really high (if B happens, A is extremely likely to occur). I believe this explains Eliezer's claim "Adding detail can make a scenario SOUND MORE PLAUSIBLE, even though the event necessarily BECOMES LESS PROBABLE" (from Burdensome Details). Essentially, I believe it sounds more plausible because you are making the wrong calculation and/or not downweighting correctly. “In fact I think that a commenter on an earlier post got it pretty much right when implying that we are implicitly calculating the probability of the story we hear given the claim rather than the probability of the claim given the story, and that this is why we get it wrong.”

In my 9/20/07 11:47 P.M. post on Burdensome details, I claimed that instead of calculating P(A and B), where B is supporting evidence of how A could happen, they are mistakenly calculating P(A given B) without downweighting the calcualtion correctly by the probability of B.

Through rearranging the definition of conditional probability, we find that
P(A and B)= P(A given B)*P(B).

It could be really easy to either forget, or to not downweight the calculation enough when P(A given B) is really high (if B happens, A is extremely likely to occur). I believe this explains Eliezer’s claim
“Adding detail can make a scenario SOUND MORE PLAUSIBLE, even though the event necessarily BECOMES LESS PROBABLE” (from Burdensome Details). Essentially, I believe it sounds more plausible because you are making the wrong calculation and/or not downweighting correctly.

]]> By: joe http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415170 joe Sat, 22 Sep 2007 19:19:30 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415170 "In fact I think that a commenter on an earlier post got it pretty much right when implying that we are implicitly calculating the probability of the story we hear given the claim rather than the probability of the claim given the story, and that this is why we get it wrong." Actually, p(story given claim)=p(claim given story). In my 9/20/07 11:47 P.M. post on Burdensome details, I claimed that instead of calculating P(A and B), where B is supporting evidence of how A could happen, they are mistakenly calculating P(A given B) without downweighting the calcualtion correctly by the probability of B. Through rearranging the definition of conditional probability, we find that P(A and B)= P(A given B)*P(B). It could be really easy to either forget, or to not downweight the calculation enough when P(A given B) is really high (if B happens, A is extremely likely to occur). I believe this explains Eliezer's claim "Adding detail can make a scenario SOUND MORE PLAUSIBLE, even though the event necessarily BECOMES LESS PROBABLE" (from Burdensome Details). Essentially, I believe it sounds more plausible because you are making the wrong calculation and/or not downweighting correctly. “In fact I think that a commenter on an earlier post got it pretty much right when implying that we are implicitly calculating the probability of the story we hear given the claim rather than the probability of the claim given the story, and that this is why we get it wrong.”

Actually, p(story given claim)=p(claim given story). In my 9/20/07 11:47 P.M. post on Burdensome details, I claimed that instead of calculating P(A and B), where B is supporting evidence of how A could happen, they are mistakenly calculating P(A given B) without downweighting the calcualtion correctly by the probability of B.

Through rearranging the definition of conditional probability, we find that
P(A and B)= P(A given B)*P(B).

It could be really easy to either forget, or to not downweight the calculation enough when P(A given B) is really high (if B happens, A is extremely likely to occur). I believe this explains Eliezer’s claim
“Adding detail can make a scenario SOUND MORE PLAUSIBLE, even though the event necessarily BECOMES LESS PROBABLE” (from Burdensome Details). Essentially, I believe it sounds more plausible because you are making the wrong calculation and/or not downweighting correctly.

]]> By: michael vassar http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415169 michael vassar Sat, 22 Sep 2007 19:03:26 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415169 I don't think that we know how to think about math probablistically with any confidence. At the very least, assigning probabilities to theories of probability or to the axioms of a probability theory seems incoherent. For the other examples, simple overconfidence is clearly involved in both the cat's eye and the Russian and Poland cases. In both cases, very casual consideration suggested no obvious way in which some statement could be true, but empirically such casual examination of a question does not justify placing high probability values on one's conclusions, as such conclusions are very frequently wrong. The cat's vision example is also, to a s substantial degree, one of pure logic and definitions, as opposed to being a factual issue. If vision is defined in such and such a manner and total darkness is defined in such and such a matter then it follows logically, e.g. p = 100%, that cats and humans are equally blind in total darkness, but we have learned nothing about the actual world by such manipulation of logical tokens unless we started with incoherent beliefs. (is part of your point that as boundedly rational beings we *do* start with incoherent beliefs? This is true, but such beliefs are fallacies, and we are biased if we fail to take this into account when assigning probabilities). I don’t think that we know how to think about math probablistically with any confidence. At the very least, assigning probabilities to theories of probability or to the axioms of a probability theory seems incoherent. For the other examples, simple overconfidence is clearly involved in both the cat’s eye and the Russian and Poland cases. In both cases, very casual consideration suggested no obvious way in which some statement could be true, but empirically such casual examination of a question does not justify placing high probability values on one’s conclusions, as such conclusions are very frequently wrong.

The cat’s vision example is also, to a s substantial degree, one of pure logic and definitions, as opposed to being a factual issue. If vision is defined in such and such a manner and total darkness is defined in such and such a matter then it follows logically, e.g. p = 100%, that cats and humans are equally blind in total darkness, but we have learned nothing about the actual world by such manipulation of logical tokens unless we started with incoherent beliefs. (is part of your point that as boundedly rational beings we *do* start with incoherent beliefs? This is true, but such beliefs are fallacies, and we are biased if we fail to take this into account when assigning probabilities).

]]> By: TGGP http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415168 TGGP Sat, 22 Sep 2007 18:58:05 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415168 In Toby's post his two claims are 1: Cats vision in complete darkness <= Human vision ICD 2: all vision ICD = 0 It cannot be the case that 2 is true but 1 is not. That makes it different from other conjunction fallacy examples. In Toby’s post his two claims are
1: Cats vision in complete darkness <= Human vision ICD
2: all vision ICD = 0
It cannot be the case that 2 is true but 1 is not. That makes it different from other conjunction fallacy examples.

]]> By: Eliezer Yudkowsky http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415167 Eliezer Yudkowsky Sat, 22 Sep 2007 18:07:47 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415167 In a previous conversation about the conjunction fallacy, I used Solomonoff induction as the background to show that if a bounded rationalist is presented with proof that a short computation halts and has a particular output, this may cause their probability estimate of that output to go up; thus being presented with the conjunction of the output and the proof may result in a higher probability being assigned than being presented with the output only. "Thus," I said, "bounded rationalists may not be able to eliminate the conjunction fallacy." But it's still a fallacy. You can't <i>really</i> have P(A&B) > P(B), said the subjective Bayesian. Incidentally, I assigned very high probability to the first proposition because of the specification of complete darkness, and then much lower probability to the second proposition because there are <i>so many</i> animals, and a universal generalization over them has so many more chances to be wrong. Think of all the burdensome extensional details implied by such a sweeping generalization! What about fireflies? What about bats? In a previous conversation about the conjunction fallacy, I used Solomonoff induction as the background to show that if a bounded rationalist is presented with proof that a short computation halts and has a particular output, this may cause their probability estimate of that output to go up; thus being presented with the conjunction of the output and the proof may result in a higher probability being assigned than being presented with the output only.

“Thus,” I said, “bounded rationalists may not be able to eliminate the conjunction fallacy.”

But it’s still a fallacy. You can’t really have P(A&B) > P(B), said the subjective Bayesian.

Incidentally, I assigned very high probability to the first proposition because of the specification of complete darkness, and then much lower probability to the second proposition because there are so many animals, and a universal generalization over them has so many more chances to be wrong. Think of all the burdensome extensional details implied by such a sweeping generalization! What about fireflies? What about bats?

]]> By: Hopefully Anonymous http://www.overcomingbias.com/2007/09/bounded-rationa.html#comment-415166 Hopefully Anonymous Sat, 22 Sep 2007 14:51:25 +0000 http://prod.ob.trike.com.au/2007/09/bounded-rationality-and-the-conjunction-fallacy.html#comment-415166 Toby, whoops, I was writing while you were posting, so my 1st paragraph concerns were pretty much answered in your 10:39am comment. Toby, whoops, I was writing while you were posting, so my 1st paragraph concerns were pretty much answered in your 10:39am comment.

]]>