Comments on: The cognitive architecture of bias http://www.overcomingbias.com/2007/01/the_cognitive_a.html Overcoming Bias is economist Robin Hanson’s blog, on honesty, signaling, disagreement, forecasting, and the far future. Fri, 19 Mar 2010 09:57:57 -0400 http://wordpress.org/?v=2.8.5 hourly 1 By: John T. Kennedy http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423111 John T. Kennedy Mon, 08 Jan 2007 00:13:08 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423111 <i>"To the extent that our scientific accounts of people's underlying motivations end up painting, as I think they will, an unflattering picture..."</i> Why unflattering? If your evaluation of your selfish nature is negative, perhaps the problem is with that evaluation rather than with that nature. “To the extent that our scientific accounts of people’s underlying motivations end up painting, as I think they will, an unflattering picture…”

Why unflattering?

If your evaluation of your selfish nature is negative, perhaps the problem is with that evaluation rather than with that nature.

]]> By: Brock http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423110 Brock Fri, 05 Jan 2007 14:25:14 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423110 Donald Davidson argued (e.g. in the last four essays in <i>Problems of Rationality</i>) that accounting for self-deception and similar forms of irrationality, such as weakness of the will, requires that the agent be divided into just such a "bureaucracy". So rather than your model for cognitive bias being an alternative model to self-deception, I'd say that what your model is a re-statement of the Davidsonian model of self-deception. Donald Davidson argued (e.g. in the last four essays in Problems of Rationality) that accounting for self-deception and similar forms of irrationality, such as weakness of the will, requires that the agent be divided into just such a “bureaucracy”.

So rather than your model for cognitive bias being an alternative model to self-deception, I’d say that what your model is a re-statement of the Davidsonian model of self-deception.

]]> By: Robin Hanson http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423109 Robin Hanson Fri, 05 Jan 2007 14:23:44 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423109 Nick, :), I was limiting myself to creatures who had beliefs and statements, as we are "sure" of that part of our illusion. So maybe we are five year old kids pretending to be press office summer interns ... Nick, :) , I was limiting myself to creatures who had beliefs and statements, as we are “sure” of that part of our illusion. So maybe we are five year old kids pretending to be press office summer interns …

]]> By: Nick Bostrom http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423108 Nick Bostrom Fri, 05 Jan 2007 02:54:14 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423108 So you think of yourself as a gullible summer intern in the McClellan press office? Illusions of grandeur, my friend. In fact, you are all like mutated E. coli bacteria on the foot of a senile louse on the head of a gullible summer intern in the McCellan press office. Take that! So you think of yourself as a gullible summer intern in the McClellan press office? Illusions of grandeur, my friend. In fact, you are all like mutated E. coli bacteria on the foot of a senile louse on the head of a gullible summer intern in the McCellan press office.

Take that!

]]> By: TGGP http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423107 TGGP Thu, 04 Jan 2007 07:02:22 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423107 The you vs your brain idea is convincingly argued against at http://www.csbmb.princeton.edu/~jdgreene/NewGreene-WebPage_files/GreeneCohenPhilTrans-04.pdf but this post was pretty good as well. Nice save with the "caricature" note, this is a really high-quality blog and it would be very upsetting if it diverged too much into politics, a realm whose bull is among the most potent fertilizers of irrationality. The you vs your brain idea is convincingly argued against at http://www.csbmb.princeton.edu/~jdgreene/NewGreene-WebPage_files/GreeneCohenPhilTrans-04.pdf but this post was pretty good as well.

Nice save with the “caricature” note, this is a really high-quality blog and it would be very upsetting if it diverged too much into politics, a realm whose bull is among the most potent fertilizers of irrationality.

]]> By: Jason Weeden http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423106 Jason Weeden Wed, 03 Jan 2007 21:35:01 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423106 Scoop, on your first question, there's a big psychological literature on self-control, ego depletion, etc., that says, yes, there are both differences between individuals in various contexts as well as differences within individuals from time to time and circumstance to circumstance. The deeper question the current research tries to get at it what it means to have *self* control or *ego* depletion in the first place. Kurban's paper, which I referenced at the beginning of the post, has a discussion on this deeper point on page 7, so I'll just point you there. I don't know about bleak, but I'd definitely say that overcoming bias is really hard, harder than most people think. The fact that it's so hard is an argument in favor of a project like Overcoming Bias, not an argument against it. If it were easy, we wouldn't have to keep thinking about it and working at it. Scoop, on your first question, there’s a big psychological literature on self-control, ego depletion, etc., that says, yes, there are both differences between individuals in various contexts as well as differences within individuals from time to time and circumstance to circumstance. The deeper question the current research tries to get at it what it means to have *self* control or *ego* depletion in the first place. Kurban’s paper, which I referenced at the beginning of the post, has a discussion on this deeper point on page 7, so I’ll just point you there.

I don’t know about bleak, but I’d definitely say that overcoming bias is really hard, harder than most people think. The fact that it’s so hard is an argument in favor of a project like Overcoming Bias, not an argument against it. If it were easy, we wouldn’t have to keep thinking about it and working at it.

]]> By: Scoop http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423105 Scoop Wed, 03 Jan 2007 21:08:32 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423105 Two questions: 1. How much does this vary from person to person? Looking at my friends and their success with New Year's resolutions, I know several people who seem pretty thoroughly controlled by their conscious selves and a lot more who are driven by unconscious forces. 2. If things are so bleak, isn't this site and nearly all attempt at introspection a waste of time? Should I abandon Overcoming Bias for porn? Two questions:
1. How much does this vary from person to person? Looking at my friends and their success with New Year’s resolutions, I know several people who seem pretty thoroughly controlled by their conscious selves and a lot more who are driven by unconscious forces.

2. If things are so bleak, isn’t this site and nearly all attempt at introspection a waste of time? Should I abandon Overcoming Bias for porn?

]]> By: William Snowmaker http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423104 William Snowmaker Wed, 03 Jan 2007 19:59:45 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423104 Actually with the right kind of introspection the press-agent believed to be "you" can be seen right through. That is the core of Zen and other approaches to end belief in the self illusion. The self is simply a persistant thought-pattern, acquired around 18 months, refined through childhood and adolescence. This self illusion is responsible for the experience of mental suffering. This is not to imply that the self illusion disappears, but its heaviness and "center of gravity" tendency evaporate once it is seen not to be real. Actually with the right kind of introspection the press-agent believed to be “you” can be seen right through.

That is the core of Zen and other approaches to end belief in the self illusion. The self is simply a persistant thought-pattern, acquired around 18 months, refined through childhood and adolescence. This self illusion is responsible for the experience of mental suffering. This is not to imply that the self illusion disappears, but its heaviness and “center of gravity” tendency evaporate once it is seen not to be real.

]]> By: Jason Weeden http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423103 Jason Weeden Wed, 03 Jan 2007 17:50:08 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423103 Robin, I'm suspicious of talk that says, you know, sure *most* people are biased, but *I'm* really just trying to get at the truth. Isn't that just what a good press agent would say? On the other hand, press agents do care about maintaining credibility, and there's the key, I think. We can construct, monitor, and enforce rules of credibility with each other and try to force honesty out of inherently dishonest systems. This is what science strives to do, though often does a terrible job at it, precisely because scientists so often warp the rules of science to fit their own selfish and tribal ends. But it's the best we've got. Hal, sure, it's an incomplete analogy. In fact, our neural press agencies do have as part of their story that they run the whole show. But it's not true. Look at the research from cognitive neuroscience and psychology -- e.g., Mike Gazzaniga's work on split-brains or Dan Wegner's work on the illusion of conscious will -- and you'll find great examples. The fact that our press agents believe, falsely, that they run the show makes self-serving biases even harder to spot and correct than they are when we clearly recognize that we're talking to a powerless spokesperson. Robin, I’m suspicious of talk that says, you know, sure *most* people are biased, but *I’m* really just trying to get at the truth. Isn’t that just what a good press agent would say? On the other hand, press agents do care about maintaining credibility, and there’s the key, I think. We can construct, monitor, and enforce rules of credibility with each other and try to force honesty out of inherently dishonest systems. This is what science strives to do, though often does a terrible job at it, precisely because scientists so often warp the rules of science to fit their own selfish and tribal ends. But it’s the best we’ve got.

Hal, sure, it’s an incomplete analogy. In fact, our neural press agencies do have as part of their story that they run the whole show. But it’s not true. Look at the research from cognitive neuroscience and psychology — e.g., Mike Gazzaniga’s work on split-brains or Dan Wegner’s work on the illusion of conscious will — and you’ll find great examples. The fact that our press agents believe, falsely, that they run the show makes self-serving biases even harder to spot and correct than they are when we clearly recognize that we’re talking to a powerless spokesperson.

]]> By: Hal Finney http://www.overcomingbias.com/2007/01/the_cognitive_a.html#comment-423102 Hal Finney Wed, 03 Jan 2007 17:30:18 +0000 http://prod.ob.trike.com.au/2007/01/the-cognitive-architecture-of-bias.html#comment-423102 That's a great story, Weeden, and I'm sure there's much truth to it. At the same time there seems to be a discrepency. The press agent does not believe that he runs the White House, yet we do believe that our conscious minds run our brains. It would be rather difficult to give Scott McClellan the impression that he ran things, yet somehow we are able to believe that about ourselves. So I tend to think that there is something missing in this analogy. That’s a great story, Weeden, and I’m sure there’s much truth to it. At the same time there seems to be a discrepency. The press agent does not believe that he runs the White House, yet we do believe that our conscious minds run our brains. It would be rather difficult to give Scott McClellan the impression that he ran things, yet somehow we are able to believe that about ourselves. So I tend to think that there is something missing in this analogy.

]]>