This entry by Eliezer struck me as an example of what I call the fallacy of the one-sided bet. As a researcher and teacher in decision analysis, I’ve noticed that this form of argument has a lot of appeal as a source of paradoxes. The key error is the framing of a situation as a no-lose (or no-win) scenario, formulating the problem in such a way so that tradeoffs are not apparent. Some examples:
"This outcome is very, very bad. In fact it's infinitely bad. But I choose it anyway because I don't care whether I get a result that's infinitely bad."
There's something peculiar about this stand but I can't quite put my finger on what it is.
Gray, are you sure one's own death has a utility to oneself ? It would seem more like a constraint on the available utility, that which exists while living. So the guy who's asked for what he would accept to die in 5 seconds is comparing a 5 second life to a longer one, not comparing a 5 second life to death.Ticking time bombs and torture : the utility cost of the torture is not in any of the more or less far fetched knock-on scenarios, it is in the destruction of our own identity, as individuals and as nations, with high costs, both internal and external. And to come back to an earlier point, those identities, for many individuals and cultures, for better or worse, are based on valuing empathy (we're back to the affect heuristic), more than utility calculations. A consideration of the sociological impact to any of the cultures which have stepped over that line within say the last century, compared to their dominant values system, would demonstrate the point.
Nick: But that's not what I was talking about, I am talking about habitual smoking vs trying one cigarette. The time scale of a 'round' is different, perhaps, but the game is the same.
g: Perhaps the utility of death is not infinite but it's certainly very large. My main point wasn't to establish consensus on how to value death (it's an important unsettled problem, after all), but to call attention to the circularity of establishing utilities of some events.
J Thomas: "Whatever your expectation, you put a finite value on it or you would never ever accept a ride with somebody who might be a bad driver."
This is precisely the circular argument I have a problem with. Perhaps people accept rides from bad drivers (on occasion) because they aren't maximizing expected utility but have a very large (or infinite) utility for dying.
Nick: It's not obvious that rationality of playing once should imply rationality of playing more than once. Consuming alcohol or smoking (or experimenting with drugs) is a good example. It may be worth it to try these things occasionally, but most people know that you don't want to consume repeatedly enough to develop problems doctors expect will develop.
The expected marginal utilities of these things are not constant - the N+1th beer has a different effect than the Nth. Drinking N beers and then stopping is still EU-maximizing.
Gray Area, I take your point about circularity, but I'm pretty sure my estimate of the value of my life isn't strictly infinite. Obviously I wouldn't agree to die in five minutes in exchange for a billion dollars, but that's because the utility-to-me of a billion dollars is dependent on how much opportunity I have to use it. There are things I'd accept a shortened life in exchange for.
On what basis do you say that the utility of death is minus infinity?
(Here's one possibility, which I think is tempting but clearly wrong: "There's nothing that I'd accept in exchange for dying in five seconds' time." Let's ignore altruism and suchlike for simplicity; with that proviso, I agree, most of us could truthfully say that. Does that mean that the negative utility we attach to dying in five seconds is infinite? Nope, it means that (again, ignoring altruism etc.) nothing *that can happen in the next five seconds* can provide us with enough utility to outweigh it. That's hardly surprising.)
_If I was torturing the guy in an attempt to stop the ticking nuke, after that situation had been resolved and he's no longer needed, I would kill him afterward and then not release the information of what happened or make up some story of how he died (suicide is a good one)._
This is old, but I just noticed the argument.
So, at one point around 10% of our prisoners at Gitmo had committed suicide. Should we believe that the US government would never follow your plan? How many of the innocent prisoners, the ones who'd cause the most trouble if released, were killed and documented as suicides?
The trouble with this reasoning is that once we start distrusting our own government we get big problems from that. The less we trust our government the less well it can protect us and the more incentive it has to lie to us, since we don't believe in it anyway. Also, if you thing the government is lying to you, that's a first step toward becoming some sort of activist and getting onto government lists and there's no telling what sort of bad thing might happen to you from there.
So it's rational to completely discount this sort of thing. Of course the US government would never be as immoral as TGGP is. We mustn't believe that could happen.
[I'm being all ironical here but there are issues involved that I haven't worked out.]
The utility of death can't be negative infinity, because people sometimes choose to die. How long can you expect to live? If you're already dying of some fatal disease and you choose to die an hour early, how much utility have you lost? An infinite amount? Only if you accept Pascal's wager and suppose that God won't like you any more if you don't see it through.
So, you have an expectation of some length of further life. A day. A year. Ten years. Seventy years. Whatever your expectation, you put a finite value on it or you would never ever accept a ride with somebody who might be a bad driver.
This is a circular argument. People figure the value of a human life is 10 million dollars BECAUSE people play Devil's Lottery games where their lives are at stake, and you assume they maximize expectations. With those assumptions, of course the value of a human life is finite, and fairly small.
A more natural assumption, in my opinion, is that the utility of death is negative and infinite (that's certainly how I think about it). Note that we use similar moral calculations where we replace 'your death' by '10 deaths' or '100 deaths.' This may be explained by the well known scope insensitivity bias, or it may be explained by utility converging to probability of win for small number of rounds (so the negative penalty is irrelevant, and consequently the number of deaths is irrelevant).
I don't think the Devils' Lotteries that are widely played every day have negative expected utility; it certainly isn't obvious that they do. I think it's pretty clear that on average aeroplane users (including the ones who have died) have benefited from aeroplane flight. Crude calculation: your probability of death on a commercial plane flight is somewhere on the order of 1 in 10 million to 1 in 50 million, depending on what sources you believe. Most attempts to work out how much value we (individually and collectively) put on human lives (our own included) end up with a figure below $10M. So: do you think the average person on a plane gets more or less than $1 of benefit from being able to make that journey? Seems pretty clear to me.
The numbers aren't quite so clear for driving, because the risk is much higher and there are more realistic alternatives, but I'd bet the expected utility comes out positive anyway.
I think you have a much better case when it comes to drinking, but that's because the probability of a bad outcome isn't so very low, which seems to me to make it not a "Devil's Lottery" regardless of the expectation.
Nick: It's not obvious that rationality of playing once should imply rationality of playing more than once. Consuming alcohol or smoking (or experimenting with drugs) is a good example. It may be worth it to try these things occasionally, but most people know that you don't want to consume repeatedly enough to develop problems doctors expect will develop.
People play the Devil's Lottery every day because most of our actions have small gains but also a small probability of death or crippling loss (e.g. driving on freeways, getting on an airplane, being exposed to carcinogens, drinking, etc.)
If it's rational to play once, it's equally rational to play once more after that, and so on ad infinitum (modulo diminishing returns of money). Sounds like the gambler's fallacy.
How do you mean people are willing to play this every day?
Devil's Lottery is a game where you spin a wheel with lots of numbers on it, and if 0 comes up you die (or a Really Bad Thing with an extremely large negative utility happens), while if any other number comes up, you get a dollar. The claim about the Devil's Lottery is that even though it may have negative expected utility (in fact arbitrarily negative), it's nevertheless rational to play as long as you don't play 'too many' rounds (e.g. enough rounds where you start to converge to the expectation).
Could you be a bit more explicit about this "Devil's Lottery"? (Is it something different from Pascal's Mugging, e.g., or do you just mean that we're constantly faced with a huge number of potential Pascal's Muggings and choose not to be mugged?)
Michael: You can't just define rationality in terms of utility expectation, you need to provide justification for why this sense of 'rational' is the right one. I take Pascal's Mugging seriously because it's the mirror image of Devil's Lottery (vast probability of small gain vs small probability of vast loss), which is a game people are willing to play every day despite its (apparent) negative expected utility. I take these sorts of games seriously enough to feel the need to define utility in a way that avoids counterintuitive results in these games, and then talk about maximizing that, rather than the conventional expectation.
To use a different line of attack: why SHOULD we use expectations without sufficient rounds to get reasonable convergence of our gain to expectations? People certainly don't in every day life, and it seems reasonable to me.
I would argue that arriving at counter-intuitive results is not an indication that ethics is non-empirical. If, after considering some argument, example, apparent paradox, or whatever, you conclude that you embrace some counter-intuitive result, you've simply discovered something about your desires which is not obvious by merely examining your instantaneous intuitive gut reactions. The various hypothetical scenarios and paradoxes serve like telescopes to discover things about your desires which are not immediately obvious.
When one works out the implications of an ethical theory and arrives at a counter-intuitive result, one of two things can happen. Sometimes the result is totally repugnant the theory is thereby falsified. In other cases, one finds that the initial gut reaction against the result is outweighed by other desires (e.g. justice, consistency, beauty) on which the theory rests. The initial reaction arose because the bearing of these desires on the scenario being considered were not obvious prior to devoting some thought to the matter.
"This outcome is very, very bad. In fact it's infinitely bad. But I choose it anyway because I don't care whether I get a result that's infinitely bad."
There's something peculiar about this stand but I can't quite put my finger on what it is.
Gray, are you sure one's own death has a utility to oneself ? It would seem more like a constraint on the available utility, that which exists while living. So the guy who's asked for what he would accept to die in 5 seconds is comparing a 5 second life to a longer one, not comparing a 5 second life to death.Ticking time bombs and torture : the utility cost of the torture is not in any of the more or less far fetched knock-on scenarios, it is in the destruction of our own identity, as individuals and as nations, with high costs, both internal and external. And to come back to an earlier point, those identities, for many individuals and cultures, for better or worse, are based on valuing empathy (we're back to the affect heuristic), more than utility calculations. A consideration of the sociological impact to any of the cultures which have stepped over that line within say the last century, compared to their dominant values system, would demonstrate the point.
Nick: But that's not what I was talking about, I am talking about habitual smoking vs trying one cigarette. The time scale of a 'round' is different, perhaps, but the game is the same.
g: Perhaps the utility of death is not infinite but it's certainly very large. My main point wasn't to establish consensus on how to value death (it's an important unsettled problem, after all), but to call attention to the circularity of establishing utilities of some events.
J Thomas: "Whatever your expectation, you put a finite value on it or you would never ever accept a ride with somebody who might be a bad driver."
This is precisely the circular argument I have a problem with. Perhaps people accept rides from bad drivers (on occasion) because they aren't maximizing expected utility but have a very large (or infinite) utility for dying.
Nick: It's not obvious that rationality of playing once should imply rationality of playing more than once. Consuming alcohol or smoking (or experimenting with drugs) is a good example. It may be worth it to try these things occasionally, but most people know that you don't want to consume repeatedly enough to develop problems doctors expect will develop.
The expected marginal utilities of these things are not constant - the N+1th beer has a different effect than the Nth. Drinking N beers and then stopping is still EU-maximizing.
Gray Area, I take your point about circularity, but I'm pretty sure my estimate of the value of my life isn't strictly infinite. Obviously I wouldn't agree to die in five minutes in exchange for a billion dollars, but that's because the utility-to-me of a billion dollars is dependent on how much opportunity I have to use it. There are things I'd accept a shortened life in exchange for.
On what basis do you say that the utility of death is minus infinity?
(Here's one possibility, which I think is tempting but clearly wrong: "There's nothing that I'd accept in exchange for dying in five seconds' time." Let's ignore altruism and suchlike for simplicity; with that proviso, I agree, most of us could truthfully say that. Does that mean that the negative utility we attach to dying in five seconds is infinite? Nope, it means that (again, ignoring altruism etc.) nothing *that can happen in the next five seconds* can provide us with enough utility to outweigh it. That's hardly surprising.)
_If I was torturing the guy in an attempt to stop the ticking nuke, after that situation had been resolved and he's no longer needed, I would kill him afterward and then not release the information of what happened or make up some story of how he died (suicide is a good one)._
This is old, but I just noticed the argument.
So, at one point around 10% of our prisoners at Gitmo had committed suicide. Should we believe that the US government would never follow your plan? How many of the innocent prisoners, the ones who'd cause the most trouble if released, were killed and documented as suicides?
The trouble with this reasoning is that once we start distrusting our own government we get big problems from that. The less we trust our government the less well it can protect us and the more incentive it has to lie to us, since we don't believe in it anyway. Also, if you thing the government is lying to you, that's a first step toward becoming some sort of activist and getting onto government lists and there's no telling what sort of bad thing might happen to you from there.
So it's rational to completely discount this sort of thing. Of course the US government would never be as immoral as TGGP is. We mustn't believe that could happen.
[I'm being all ironical here but there are issues involved that I haven't worked out.]
The utility of death can't be negative infinity, because people sometimes choose to die. How long can you expect to live? If you're already dying of some fatal disease and you choose to die an hour early, how much utility have you lost? An infinite amount? Only if you accept Pascal's wager and suppose that God won't like you any more if you don't see it through.
So, you have an expectation of some length of further life. A day. A year. Ten years. Seventy years. Whatever your expectation, you put a finite value on it or you would never ever accept a ride with somebody who might be a bad driver.
This is a circular argument. People figure the value of a human life is 10 million dollars BECAUSE people play Devil's Lottery games where their lives are at stake, and you assume they maximize expectations. With those assumptions, of course the value of a human life is finite, and fairly small.
A more natural assumption, in my opinion, is that the utility of death is negative and infinite (that's certainly how I think about it). Note that we use similar moral calculations where we replace 'your death' by '10 deaths' or '100 deaths.' This may be explained by the well known scope insensitivity bias, or it may be explained by utility converging to probability of win for small number of rounds (so the negative penalty is irrelevant, and consequently the number of deaths is irrelevant).
I don't think the Devils' Lotteries that are widely played every day have negative expected utility; it certainly isn't obvious that they do. I think it's pretty clear that on average aeroplane users (including the ones who have died) have benefited from aeroplane flight. Crude calculation: your probability of death on a commercial plane flight is somewhere on the order of 1 in 10 million to 1 in 50 million, depending on what sources you believe. Most attempts to work out how much value we (individually and collectively) put on human lives (our own included) end up with a figure below $10M. So: do you think the average person on a plane gets more or less than $1 of benefit from being able to make that journey? Seems pretty clear to me.
The numbers aren't quite so clear for driving, because the risk is much higher and there are more realistic alternatives, but I'd bet the expected utility comes out positive anyway.
I think you have a much better case when it comes to drinking, but that's because the probability of a bad outcome isn't so very low, which seems to me to make it not a "Devil's Lottery" regardless of the expectation.
Nick: It's not obvious that rationality of playing once should imply rationality of playing more than once. Consuming alcohol or smoking (or experimenting with drugs) is a good example. It may be worth it to try these things occasionally, but most people know that you don't want to consume repeatedly enough to develop problems doctors expect will develop.
People play the Devil's Lottery every day because most of our actions have small gains but also a small probability of death or crippling loss (e.g. driving on freeways, getting on an airplane, being exposed to carcinogens, drinking, etc.)
If it's rational to play once, it's equally rational to play once more after that, and so on ad infinitum (modulo diminishing returns of money). Sounds like the gambler's fallacy.
How do you mean people are willing to play this every day?
Devil's Lottery is a game where you spin a wheel with lots of numbers on it, and if 0 comes up you die (or a Really Bad Thing with an extremely large negative utility happens), while if any other number comes up, you get a dollar. The claim about the Devil's Lottery is that even though it may have negative expected utility (in fact arbitrarily negative), it's nevertheless rational to play as long as you don't play 'too many' rounds (e.g. enough rounds where you start to converge to the expectation).
Could you be a bit more explicit about this "Devil's Lottery"? (Is it something different from Pascal's Mugging, e.g., or do you just mean that we're constantly faced with a huge number of potential Pascal's Muggings and choose not to be mugged?)
Sorry for the delayed reply.
Michael: You can't just define rationality in terms of utility expectation, you need to provide justification for why this sense of 'rational' is the right one. I take Pascal's Mugging seriously because it's the mirror image of Devil's Lottery (vast probability of small gain vs small probability of vast loss), which is a game people are willing to play every day despite its (apparent) negative expected utility. I take these sorts of games seriously enough to feel the need to define utility in a way that avoids counterintuitive results in these games, and then talk about maximizing that, rather than the conventional expectation.
To use a different line of attack: why SHOULD we use expectations without sufficient rounds to get reasonable convergence of our gain to expectations? People certainly don't in every day life, and it seems reasonable to me.
I would argue that arriving at counter-intuitive results is not an indication that ethics is non-empirical. If, after considering some argument, example, apparent paradox, or whatever, you conclude that you embrace some counter-intuitive result, you've simply discovered something about your desires which is not obvious by merely examining your instantaneous intuitive gut reactions. The various hypothetical scenarios and paradoxes serve like telescopes to discover things about your desires which are not immediately obvious.
When one works out the implications of an ethical theory and arrives at a counter-intuitive result, one of two things can happen. Sometimes the result is totally repugnant the theory is thereby falsified. In other cases, one finds that the initial gut reaction against the result is outweighed by other desires (e.g. justice, consistency, beauty) on which the theory rests. The initial reaction arose because the bearing of these desires on the scenario being considered were not obvious prior to devoting some thought to the matter.
Ethics is not an empirical science because intuitions are not universal and ethics as practiced often embraces counter-intuitive results.