Comments on: The Bad Guy Bias http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html Overcoming Bias is economist Robin Hanson’s blog, on honesty, signaling, disagreement, forecasting, and the far future. Sat, 11 Feb 2012 23:23:58 +0000 hourly 1 http://wordpress.org/?v=3.1.3 By: John David Galt http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390828 John David Galt Sun, 14 Dec 2008 18:48:16 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390828 This bias is easily explained: we expect (in the sense of "demand") people to treat each other decently. One can't be sapient without having this duty. Thus, people close to the victim of a killer or rapist quite reasonably believe there's no excuse for the crime to have occurred -- a specific person is responsible and needs to be punished and/or made to pay restitution, so feeling outrage toward him/her is the only appropriate response. In contrast, car wrecks (excluding those caused deliberately or by gross dereliction) are sufficiently unforseeable (and the precautions that might have prevented them are sufficiently non-worthwhile) that there is nobody to blame: such things inevitably happen to a few. The same reasoning applies to lung cancer, if we credit the victim's own decision that smoking is enjoyable enough to be worth the risk to him/her. (If we don't, then it is self-inflicted.) In either case there is no reasonable target for outrage, except maybe "God". This bias is easily explained: we expect (in the sense of “demand”) people to treat each other decently. One can’t be sapient without having this duty. Thus, people close to the victim of a killer or rapist quite reasonably believe there’s no excuse for the crime to have occurred — a specific person is responsible and needs to be punished and/or made to pay restitution, so feeling outrage toward him/her is the only appropriate response.

In contrast, car wrecks (excluding those caused deliberately or by gross dereliction) are sufficiently unforseeable (and the precautions that might have prevented them are sufficiently non-worthwhile) that there is nobody to blame: such things inevitably happen to a few. The same reasoning applies to lung cancer, if we credit the victim’s own decision that smoking is enjoyable enough to be worth the risk to him/her. (If we don’t, then it is self-inflicted.) In either case there is no reasonable target for outrage, except maybe “God”.

]]> By: Hopefully Anonymous http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390827 Hopefully Anonymous Thu, 11 Dec 2008 09:41:32 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390827 subsidiary to repugnancy bias and commission bias (bad things committed by an entity with apparent agency is more repugnant). subsidiary to repugnancy bias and commission bias (bad things committed by an entity with apparent agency is more repugnant).

]]> By: luzr http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390826 luzr Wed, 10 Dec 2008 06:45:07 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390826 Hal: "perhaps time spent aiming to avoid disasters involving super-intelligence might be better spent working to improve institutions to prevent war" I would go even further saying that perhaps Strong AGI is the institution to prevent the war and other disasters. There are so many risks to human civilization and most of them can be prevented in posthumanity environment... Nancy: "Aging is probably the biggest non-badguy damage happening in the civilized world." Yep. Another case for moving things ahead. Hal:

“perhaps time spent aiming to avoid disasters involving super-intelligence might be better spent working to improve institutions to prevent war”

I would go even further saying that perhaps Strong AGI is the institution to prevent the war and other disasters.

There are so many risks to human civilization and most of them can be prevented in posthumanity environment…

Nancy:

“Aging is probably the biggest non-badguy damage happening in the civilized world.”

Yep. Another case for moving things ahead.

]]> By: Nancy Lebovitz http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390825 Nancy Lebovitz Tue, 09 Dec 2008 23:44:31 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390825 Aging is probably the biggest non-badguy damage happening in the civilized world. Aging is probably the biggest non-badguy damage happening in the civilized world.

]]> By: Hal Finney http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390824 Hal Finney Tue, 09 Dec 2008 22:15:20 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390824 That's a good question, Nancy; I was wondering about the same thing. It is certainly possible that warfare will turn out to be the overwhelmingly biggest disaster facing humanity in the 21st century, and that perhaps time spent aiming to avoid disasters involving super-intelligence might be better spent working to improve institutions to prevent war. Of course, catastrophic future wars, armageddon, post-apocalyptic survival, etc, are among the oldest tropes in speculative fiction. So I'm not sure that the bias here applies, that we under-estimate the impact of war. Eliezer, minds do have a disproportionately large impact, but at the same time, historically humanity has arguably been more harmed by non-minds than minds. This bias makes us focus too hard on the actions of minds. I am trying to think of what future problems we might be ignoring because of this bad-guy bias. I suppose it comes down to the kinds of things Vedantam mentions, natural disasters and accidents. Maybe the real key to happiness and prosperity in the future is encouraging more people to buy insurance. That will not only protect them against certain harms, it will send price signals to avoid risky behavior. That’s a good question, Nancy; I was wondering about the same thing. It is certainly possible that warfare will turn out to be the overwhelmingly biggest disaster facing humanity in the 21st century, and that perhaps time spent aiming to avoid disasters involving super-intelligence might be better spent working to improve institutions to prevent war. Of course, catastrophic future wars, armageddon, post-apocalyptic survival, etc, are among the oldest tropes in speculative fiction. So I’m not sure that the bias here applies, that we under-estimate the impact of war.

Eliezer, minds do have a disproportionately large impact, but at the same time, historically humanity has arguably been more harmed by non-minds than minds. This bias makes us focus too hard on the actions of minds.

I am trying to think of what future problems we might be ignoring because of this bad-guy bias. I suppose it comes down to the kinds of things Vedantam mentions, natural disasters and accidents. Maybe the real key to happiness and prosperity in the future is encouraging more people to buy insurance. That will not only protect them against certain harms, it will send price signals to avoid risky behavior.

]]> By: Jayson Virissimo http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390823 Jayson Virissimo Tue, 09 Dec 2008 21:57:47 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390823 "these populations still show bad guy bias, even to the point of explicit beliefs that it's somehow worse to be killed than to die in other similarly violent ways" -michael vassar Michael, are you saying you shouldn't be more upset if I kicked you in the shin than if you just bumped your shin on a railing? “these populations still show bad guy bias, even to the point of explicit beliefs that it’s somehow worse to be killed than to die in other similarly violent ways” -michael vassar

Michael, are you saying you shouldn’t be more upset if I kicked you in the shin than if you just bumped your shin on a railing?

]]> By: Nancy Lebovitz http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390822 Nancy Lebovitz Tue, 09 Dec 2008 21:28:46 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390822 I think war gets treated as more like a natural disaster than the result of ill intent. The number of Jews killed in the Holocaust is quite well known. The number of non-Jews killed in the Holocaust is fairly well known. The number of people killed by Hitler's invasions is generally not mentioned. I think war gets treated as more like a natural disaster than the result of ill intent. The number of Jews killed in the Holocaust is quite well known. The number of non-Jews killed in the Holocaust is fairly well known. The number of people killed by Hitler’s invasions is generally not mentioned.

]]> By: Eliezer Yudkowsky http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390821 Eliezer Yudkowsky Tue, 09 Dec 2008 21:08:29 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390821 I could make the same point with respect to unFriendly AI; the ones you have to worry about aren't "evil" in the sense of carrying out deliberately malevolent deeds, but rather, the ones who don't care about your existence one way or another (and you are made of atoms that they could use for things they do care about). Minds are more powerful and have a larger impact than just about anything else; smarter minds have larger impacts. People concerned with large impacts on the future have a natural cause to be concerned about extremely intelligent actors. And for obvious reasons, we shouldn't be worrying about those whose desires approach our own reflective equilibria, but rather those which are neutral, twisted-good, or (in the case of augmented uploads) evil. I could make the same point with respect to unFriendly AI; the ones you have to worry about aren’t “evil” in the sense of carrying out deliberately malevolent deeds, but rather, the ones who don’t care about your existence one way or another (and you are made of atoms that they could use for things they do care about).

Minds are more powerful and have a larger impact than just about anything else; smarter minds have larger impacts. People concerned with large impacts on the future have a natural cause to be concerned about extremely intelligent actors. And for obvious reasons, we shouldn’t be worrying about those whose desires approach our own reflective equilibria, but rather those which are neutral, twisted-good, or (in the case of augmented uploads) evil.

]]> By: michael vassar http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390820 michael vassar Tue, 09 Dec 2008 19:35:50 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390820 Robin: Only in the sense that evolution or economic attractors are ill intentions. It's VERY hard to avoid anthropomorphizing AGI unless you keep those in mind as part of your default sample set of "optimizers". Robin: Only in the sense that evolution or economic attractors are ill intentions.
It’s VERY hard to avoid anthropomorphizing AGI unless you keep those in mind as part of your default sample set of “optimizers”.

]]> By: Hal Finney http://www.overcomingbias.com/2008/12/the-bad-guy-bia.html#comment-390819 Hal Finney Tue, 09 Dec 2008 18:33:06 +0000 http://prod.ob.trike.com.au/2008/12/the-bad-guy-bias.html#comment-390819 Right, I think the point at the end is that this bias makes us too concerned about unfriendly AI, because it fits into this "evil actor" model. One might also point to the many criticisms raised here over Robin's "<a href="http://www.overcomingbias.com/2008/11/emulations-go-f.html" rel="nofollow">ems</a>" scenario along these lines, that the ems would be evil (or at least selfish and/or uncaring) and do things that would wipe out the human race or worse. Both of these concerns direct us to focus on the motivations of intelligent and powerful beings as a principal threat to our happiness. This framework is much the same as how people today view the threat of terrorism. Even more naturalistic threats, like global warming or resource exhaustion, tend to be interpreted in a model based on "bad guy" actions. It's nobody's fault, really, that the world may be running out of oil, or capacity to absorb CO2 - it's just an unfortunate fact of nature. Kind of an odd coincidence that it is happening just as a demographic transition is reducing population growth rates, as well as when technology is almost there for cheap work-arounds to these problems. If the world had had one order of magnitude more capacity for these resources then we'd probably have a much smoother time transitioning. But the issues are often framed such that we look for someone to blame, we see the problem as a result of bad behavior. Right, I think the point at the end is that this bias makes us too concerned about unfriendly AI, because it fits into this “evil actor” model. One might also point to the many criticisms raised here over Robin’s “ems” scenario along these lines, that the ems would be evil (or at least selfish and/or uncaring) and do things that would wipe out the human race or worse. Both of these concerns direct us to focus on the motivations of intelligent and powerful beings as a principal threat to our happiness. This framework is much the same as how people today view the threat of terrorism.

Even more naturalistic threats, like global warming or resource exhaustion, tend to be interpreted in a model based on “bad guy” actions. It’s nobody’s fault, really, that the world may be running out of oil, or capacity to absorb CO2 – it’s just an unfortunate fact of nature. Kind of an odd coincidence that it is happening just as a demographic transition is reducing population growth rates, as well as when technology is almost there for cheap work-arounds to these problems. If the world had had one order of magnitude more capacity for these resources then we’d probably have a much smoother time transitioning. But the issues are often framed such that we look for someone to blame, we see the problem as a result of bad behavior.

]]>