In addition to (allegedly) scope insensitivity and "motivated continuation," I would like to suggest that the incredibly active discussion on the torture vs. specks post is also driven in part by a bias toward, well, toward closure; a bias toward determinate answers: a bias toward decision procedures that are supposed to yield an answer in every case, and one that can be implemented by humans in the world in which we live and with the biological and social pressures that we face.
There's a central question that I think it is useful to address directly to some extent for this discussion: what is morality? What does it mean for a choice to be "immoral"? This is a question of definition really, so the answer is somewhat subjective.
I think "moral" and "immoral" are complex terms in themselves: almost(?) anyone could increase their net benefit to others, if they made personal sacrifices to do so, as most people do not have as their foremost goal to maximize their benefit to others at all times. Is it "immoral" not to hold altruism as the goal of highest priority in all cases? It is merely less moral than the highest possible degree of morality. I think it is usually more useful to speak of morality of choices relative to their alternatives, than by using vaguely-defined thresholds for a "moral" choice versus an "immoral" choice - language which I think can lead to a lot of confusion.
Morality, to me, refers to how one's actions affect one's expectation for another's expected utility. That is, in a simple world with only Alice and Bob, it is immoral for Alice to act in a way that she would expect to reduce (Bob's expected utility, according to Bob). The morality of Alice's choice X can be described thus:
Ma: Morality for AliceX: a certain optionPa: Alice's perceptionUb: Bob's expected utilityUba: Bob's expected utility, as Alice would predict it
Ma(X) = Uba(X) = Pa(Ub(X))
Though this quantitative view is a simple model that provides a way of analysing decisions from a moral standpoint, it still does not make answering moral question easy: the difficult part is counting not just Alice's expectation of the effect of a decision on Bob, but on everyone affected (including Alice) - a decision's overall morality is clearly determined by its effects on everyone affected.
It would seem that summing one's expected effect on all people involved is the proper way to determine overall morality, as such a system has the desirable property that a decision that affects ten bystanders in a certain way is morally equivalent to a smaller decision that affects each of the ten people individually, in the same way.
In order to perform such a summation, it is necessary to normalize other people's expected utilities - or equivalently, to measure them all in absolute, interchangeable units of effect. While this is impossible to do perfectly, I think it is quite possible to approximate.
Note that this morality function is not affected by benefit or harm to the decision-maker. This may seem like a flaw; is it not more moral for Alice to harm Bob slightly if she must do so to receive some great benefit, than it is for Alice to do the same harm for very little benefit to herself? No, I think it is not; this kind of decision generally exposes the value Alice places on morality: in the former situation, Alice may make the decision even if she places fairly a high price on morality, while the latter situation suggests that Alice is willing to take immoral action for a lower price - the morality of the choice is not affected by such context, though what it may reveal about Alice's morality is (and the two are easily confused).
One possible objection to this system would be the idea that this would mean that it is morally neutral to help Hitler achieve his goals at the expense of hindering Gandhi, each to such an extent that their expected utilities are affected by the same (but opposite) amount. Not so! This quantitative approach requires taking into account all foreseeable effects, including indirect effects... which is part of the reason moral questions aren't easy.
So I agree that moral questions aren't easy, but I don't think it's because it's difficult to reason about morality abstractly; in my opinion the difficulty is mostly in forecasting the effects of one's decisions, which is a problem squarely in the realm of decision theory - and thus the methods of doing so are well studied.
I would put Paul's question another way. If, as many people (including myself) believe that what we call moral systems are actually rules of thumb generated by genetic machinery that evolved to ensure group survival in the early days of humanity, then we should not expect to be able to do any moral calculus. The moral rules of thumb we have were not designed to be consistent or "right" in any fundamental sense, they are there only to ensure that the group worked together, rules that didn't work didn't survive. As a result, in todays modern world, where we are faced with many different situations that our ancestors were not faced with, it is to be expected that we will encounter many moral paradoxes. If genetic rules of thumbs are really where morality originates it will be especially easy (trivial) to generate hypothetical examples of moral paradoxes, or difficult moral calculations, as we have seen on this blog.
On the question of whether decision theory is useful in moral calculations, I would say there is an analogy with economics, where the utility question is simplified to be optimisation of a particular chosen variable, usually money. Once you have decided to optimise that variable, then a calculation becomes possible. But economics has no say in whether one variable should be optimised over another one.
Bob said: RE Nick's comment, think about the implications of your claim. It's actually worse - if everyone is being tortured and you can save all of them but a single person, that is not an improvement?
It's not an improvement for the person who is still being tortured.
If you've managed to save some people from being tortured, you have indeed done a good thing for those specific people.
You haven't "done a good thing" in the abstract, utilitarian sense of "maximizing the greater good" from some imaginary omniscient perspective, because that perspective doesn't exist.
I do think we should try to save as many people from torture as possible, and I realize that in the practical sense, it might not be possible to save everyone all the time. But that doesn't make the "one guy is being tortured" situation morally acceptable, or more morally acceptable than the "everyone is being tortured" situation.
Both situations are equally unacceptable, because in each case, there is torture (which, again, is superlatively awful by design) being experienced from an individual perspective.
Perhaps if you're the tortured individual, and you're aware that everyone else has been saved, you might feel better from an empathic standpoint. But it seems doubtful that torturers would want to tell their victim anything that might make him feel better.
I guess what I'm trying to say is, acknowledging the practical limitations of one's rescue tactics doesn't equate to having achieved moral acceptability or superiority. I also think that these hypothetical zero-sum-game dilemmas are unrealistic. In real life, the choice is not usually really between "saving 1 person vs. saving 10 people" -- and I think that people who train themselves to think in "zero-sum terms" risk stifling their creative faculties for dealing with difficult situations.
RE Nick's comment, think about the implications of your claim. It's actually worse - if everyone is being tortured and you can save all of them but a single person, that is not an improvement?
And I'm sure answers to this would vary as well -- personally I see torturing anyone as just as bad as torturing everyone, but that's beside the point right now.
If you save one person from torture, but there are other people being tortured that you can't do anything about, have you not done something good? I don't think this is beside the point - part of the original dispute is to what extent suffering is additive.
My first thought was "specks" and after further internal deliberation and reading through the comments, I find myself at the same conclusion. The attempt to compare "specks" with "torture" in this manner is incoherent. It's probably reasonable to assume that practically everyone gets a speck of dust in one or both of their eyes on a daily basis. Dust exists pretty much everywhere humans exist, and the probability of spending your day anywhere outside an industrial cleanroom *without* having dust contact your ocular region is likely very low.
So it's not as if our present position is one of zero knowledge regarding the effect of dust specks in our eyes -- this is something that happens in the real world, every day, and honestly, it's not something that garners a lot of complaint. Torture, however, is superlatively awful *by design*. And I personally would rather live in a world where dust specks were commonplace, but nobody was being tortured, than the reverse.
How a person answers this question probably depends somewhat on how literally they take it. I immediately considered a situation consisting of a comparison between actual dust specks and actual torture, and my analysis was informed by that literalness. However, it seems that some respondents interpreted the question in a more purely abstract sense -- e.g., rather than applying their ethical sense to the real world, they approached the problem as one of getting the "right" answer from a consciously algorithmic standpoint.
I am not getting why some seem tempted to invoke the viewpoint of an entity capable of somehow "feeling" the cumulative effect of tiny amounts of possible badness. If no such entity exists to experience this aggregate suffering, then it doesn't make sense to suggest that imagining an aggregate suffering made up of a zillion teensy annoyances is the best way to reason out one's moral decisions.
Also, the fact that different people are responding to this dilemma in different ways (and with different reasoning paths) hopefully demonstrates that there's more than one way to consider such questions, and that no, there's not likely to be a One True Algorithm that somehow allows a person to make "proper" moral decisions quickly and tidily in every case. A memorized "abstraction machine" cannot be used as a substitute for actual thinking, of the sort that engages directly with the relevant, real, data at hand in a given situation. Sure, abstraction machines can help a person structure their thoughts in some cases, but I firmly believe that people shouldn't let their devotion to keeping their "isms" harmonious override their devotion to the well-being of individuals.
With regard to the original question again, I think the point might have better been served by simply asking whether torturing one person or torturing a whole lot of people was worse. (And I'm sure answers to this would vary as well -- personally I see torturing anyone as just as bad as torturing everyone, but that's beside the point right now).
The "dust specks" example was so inane as to be functionally meaningless in discussion -- you might as well have said, "What would be worse, torturing one person for 50 years, or giving 3^^^3 people a stuffed toy unicorn?" Given that stuffed toy unicorns are bigger than dust specks, and potentially more dangerous (e.g., people could trip over them!), it would seem that there's a greater Existential Unicorn Risk than an Existential Dust Speck Risk. But I'd still choose "Unicorns for All" world over "Torture One Guy" world any day.
I'm still struggling with how tiny dispersed ouches don't *eventually* add up to one big ouch. The only way is if they represent zero harm. Personally, this was my first line of defense for my specks intuition but I decided that the question precluded the harm from actually being zero. In reality, I believe that there is a threshold of suffering below which there is no harm. Consider, perhaps, the level of harm you would be willing to suffer for a 1/N probability (N very large but not too large) that you would save a stranger's life. If you willingly accept harm that has no benefit to you and almost no expected benefit to anyone, can we really call it harm? The specks seem like they would clearly meet this test.
Of course, to rcriii's post, I'm not trying to make this a realistic question. And assuming nonzero harm, it has to aggregate somehow. Nick raises an interesting idea but what would make us believe that the disutility of harm is nonlinear enough to avoid the problem as the number of tiny harms goes to infinity?
Bob, I have to chime in here an say that the original question was _not_ carefully posed:
- Very early in the thread someone noted that 3^^^3 is much greater than the likely number of humans ever. So we are asked to decide between something that has happened in the past and an impossibility.- Elezier did not put any kind of value on the harm caused by the 'speck'. Is it 50/1^^^3 of 1 year of torture, more or less? How can we do any kind of 'calculus' if we don't know?- How do we account for the fact that a speck once washed or brushed out on the eye is quickly forgotten, but torture leaves physical and psychological scars that can last a lifetime? Or are these memorable specks?
Mike, I wasn't claiming equivalence between my question and Eliezer's, only trying to make the point that it's harder to judge under uncertainty if you have incommensurable values.
If 3^^^3 people actually existed, then a 1/3^^^3 probability would not be trivial in absolute terms - for instance, torturing each existing person with probability 1/3^^^3 would be very bad, while it's ridiculously trivial in our world.
Poke 5000 people with one thumbtack each, or poke 1 person 4999 times. I think it's still reasonable to believe that tiny dispersed ouch just doesn't add up to one big ouch.
I agree, but not because they're somehow incommensurable; rather, suffering is linear with single pokes across multiple people but greater than linear when the same person is poked more than once, because of emotional effects on top of the raw pain from the pokes. I would indeed say there is some N between 1 and 5000 where N pokes to 1 person goes from better to worse than 1 poke to 5000 people, but I don't see it as the "magic poke"; it's just the point where a certain increasing nonlinear function crosses a certain constant value. In practice, we don't and can't know the function or the value, so we generally have to decide on principle, and this principle may include discrete "ouchie" and "brutalization" categories; but that doesn't mean they're not both "just" suffering.
Bob: Ok, I'll go along with that. Let's abstract from torture and specks and just say "ouch." Poke 5000 people with one thumbtack each, or poke 1 person 4999 times. I think it's still reasonable to believe that tiny dispersed ouch just doesn't add up to one big ouch. It just doesn't aggregate like that. At some point there's a "magic poke" where you're not just having ouchies, you're being brutalized. And no, I can't say where the number is, but if morality isn't decision theoretic, must I do so?
Nick: To be consistent, I'd have to go with the specking in your case too. But is consistency required? (My intuition about that example is "who cares," because the torture is so uncertain that it can be discounted to zero as long as we don't repeat it.)
Nick:Paul: How, then, do you decide between certain specks and uncertain harm? Would you rather speck 3^^^3 people with probability 1 or torture one person with probability 1/3^^^3?As I mentioned in the other thread, that question and Eliezer's question are profoundly different. In Eliezer's example, you are torturing someone with 100% certainty, at the cost of a trivial harm. (My answer to Eliezer's question is essentially that (trivial harm)*(3^^^3 people)=trivial harm). In you're example, you are choosing a trivial harm against against a trivial (basically=0) probability that someone will get tortured.
3^^^3 people simply cannot be equated with a 1/3^^^3 probability, which, if intelligible at all, is equal to 0 with greater certainty by far than anything else we are "most certain" about in the real world.
"But I truly don't see how anyone can stick with specks under what seems to be the most obvious interpretation of the original question."
The problem here is analogous with the "paradox" of the iterated Prisoners' Dilemma. If the consequences can be completely specified then the problem is one of classical rational decision-making. But in the real-world, any agent is constrained by a point of view that is both (1) located, and (2) limited. For this reason, our best approach to this interesting and realistic class of problems is to implement solutions based on best known principles (and wait to see the actual complex outcome.)
It's like being faced with the problem of a huge chasm that separates one from trading partners on the other side. Does one assume that a bridge is needed, and proceed to allocate physical and intellectual resources to building the best possible bridge? Or does one focus on defining the problem in terms of (1) one's values and (2) best known principles of promoting those values? In the latter case, the process of defining and refining the problem actually leads to defining the solution, which may turn out to be not a bridge per se but perhaps a more highly adapted system of communication and delivery of goods.
Well, I've reached my self-imposed quota for the day, despite this being an interesting topic and one that is crucial to optimizing humanity's path toward an increasingly complex future.
Paul: How, then, do you decide between certain specks and uncertain harm? Would you rather speck 3^^^3 people with probability 1 or torture one person with probability 1/3^^^3?
Paul, but the original question was very carefully posed as different degrees of harm. I share your desire to elevate torture as a different sort of bad. I just can't see how to do it in a consistent way without changing the question. And I think that your (really "our") desire to change the question is one of the more interesting things to come out of the discussion.
"But I'm not arguing that uncalculability comes from subjectivity. While my eventual choice between the resistance or the duty to mother will have to be subjective (that's why it's an expression of character), the problem itself need not have anything inherently subjective about it."
Moral decision-making always entails two fundamental elements: the agent's (1) model of a subjectively desired state, and (2) model of (increasingly) objective principles of effective interaction with 'reality'. Each is crucial to mensuration of matters of morality. Each is differently difficult.
As for the Trolley Problem, it doesn't show that morality is mysterious, but only that the popular conception of morality is flawed. Humans' evaluation of moral issues is encoded at various levels, from innate drives including disgust or pride, to cultural drives expressed by societal/religious norms, to logical reasoning about maximizing expected utility. It's not surprising that differing representations of the problem yield differing solutions, nor that we feel so strongly about questions of what is "right."
Each of these models has in common that they encode, not "what is right" in any objective way, but rather evolutionary "wisdom" of "what works" in principle to further the values of an increasing context of agents over an increasing scope of consequences. We, being a result of these same evolutionary processes, therefore tend to see our branch of "what increasingly works" as isomorphic with "increasingly right."
The practical significance of this is that it takes us a step beyond shallow utilitarianism, expanding our intentional focus from perceived desired consequences in an inherently uncertain future, to *improving the process* that delivers increasingly desirable consequences via increasing effective awareness of elements 1 and 2 above.
There's a central question that I think it is useful to address directly to some extent for this discussion: what is morality? What does it mean for a choice to be "immoral"? This is a question of definition really, so the answer is somewhat subjective.
I think "moral" and "immoral" are complex terms in themselves: almost(?) anyone could increase their net benefit to others, if they made personal sacrifices to do so, as most people do not have as their foremost goal to maximize their benefit to others at all times. Is it "immoral" not to hold altruism as the goal of highest priority in all cases? It is merely less moral than the highest possible degree of morality. I think it is usually more useful to speak of morality of choices relative to their alternatives, than by using vaguely-defined thresholds for a "moral" choice versus an "immoral" choice - language which I think can lead to a lot of confusion.
Morality, to me, refers to how one's actions affect one's expectation for another's expected utility. That is, in a simple world with only Alice and Bob, it is immoral for Alice to act in a way that she would expect to reduce (Bob's expected utility, according to Bob). The morality of Alice's choice X can be described thus:
Ma: Morality for AliceX: a certain optionPa: Alice's perceptionUb: Bob's expected utilityUba: Bob's expected utility, as Alice would predict it
Ma(X) = Uba(X) = Pa(Ub(X))
Though this quantitative view is a simple model that provides a way of analysing decisions from a moral standpoint, it still does not make answering moral question easy: the difficult part is counting not just Alice's expectation of the effect of a decision on Bob, but on everyone affected (including Alice) - a decision's overall morality is clearly determined by its effects on everyone affected.
It would seem that summing one's expected effect on all people involved is the proper way to determine overall morality, as such a system has the desirable property that a decision that affects ten bystanders in a certain way is morally equivalent to a smaller decision that affects each of the ten people individually, in the same way.
In order to perform such a summation, it is necessary to normalize other people's expected utilities - or equivalently, to measure them all in absolute, interchangeable units of effect. While this is impossible to do perfectly, I think it is quite possible to approximate.
Note that this morality function is not affected by benefit or harm to the decision-maker. This may seem like a flaw; is it not more moral for Alice to harm Bob slightly if she must do so to receive some great benefit, than it is for Alice to do the same harm for very little benefit to herself? No, I think it is not; this kind of decision generally exposes the value Alice places on morality: in the former situation, Alice may make the decision even if she places fairly a high price on morality, while the latter situation suggests that Alice is willing to take immoral action for a lower price - the morality of the choice is not affected by such context, though what it may reveal about Alice's morality is (and the two are easily confused).
One possible objection to this system would be the idea that this would mean that it is morally neutral to help Hitler achieve his goals at the expense of hindering Gandhi, each to such an extent that their expected utilities are affected by the same (but opposite) amount. Not so! This quantitative approach requires taking into account all foreseeable effects, including indirect effects... which is part of the reason moral questions aren't easy.
So I agree that moral questions aren't easy, but I don't think it's because it's difficult to reason about morality abstractly; in my opinion the difficulty is mostly in forecasting the effects of one's decisions, which is a problem squarely in the realm of decision theory - and thus the methods of doing so are well studied.
I would put Paul's question another way. If, as many people (including myself) believe that what we call moral systems are actually rules of thumb generated by genetic machinery that evolved to ensure group survival in the early days of humanity, then we should not expect to be able to do any moral calculus. The moral rules of thumb we have were not designed to be consistent or "right" in any fundamental sense, they are there only to ensure that the group worked together, rules that didn't work didn't survive. As a result, in todays modern world, where we are faced with many different situations that our ancestors were not faced with, it is to be expected that we will encounter many moral paradoxes. If genetic rules of thumbs are really where morality originates it will be especially easy (trivial) to generate hypothetical examples of moral paradoxes, or difficult moral calculations, as we have seen on this blog.
On the question of whether decision theory is useful in moral calculations, I would say there is an analogy with economics, where the utility question is simplified to be optimisation of a particular chosen variable, usually money. Once you have decided to optimise that variable, then a calculation becomes possible. But economics has no say in whether one variable should be optimised over another one.
Bob said: RE Nick's comment, think about the implications of your claim. It's actually worse - if everyone is being tortured and you can save all of them but a single person, that is not an improvement?
It's not an improvement for the person who is still being tortured.
If you've managed to save some people from being tortured, you have indeed done a good thing for those specific people.
You haven't "done a good thing" in the abstract, utilitarian sense of "maximizing the greater good" from some imaginary omniscient perspective, because that perspective doesn't exist.
I do think we should try to save as many people from torture as possible, and I realize that in the practical sense, it might not be possible to save everyone all the time. But that doesn't make the "one guy is being tortured" situation morally acceptable, or more morally acceptable than the "everyone is being tortured" situation.
Both situations are equally unacceptable, because in each case, there is torture (which, again, is superlatively awful by design) being experienced from an individual perspective.
Perhaps if you're the tortured individual, and you're aware that everyone else has been saved, you might feel better from an empathic standpoint. But it seems doubtful that torturers would want to tell their victim anything that might make him feel better.
I guess what I'm trying to say is, acknowledging the practical limitations of one's rescue tactics doesn't equate to having achieved moral acceptability or superiority. I also think that these hypothetical zero-sum-game dilemmas are unrealistic. In real life, the choice is not usually really between "saving 1 person vs. saving 10 people" -- and I think that people who train themselves to think in "zero-sum terms" risk stifling their creative faculties for dealing with difficult situations.
AnneC,
RE Nick's comment, think about the implications of your claim. It's actually worse - if everyone is being tortured and you can save all of them but a single person, that is not an improvement?
And I'm sure answers to this would vary as well -- personally I see torturing anyone as just as bad as torturing everyone, but that's beside the point right now.
If you save one person from torture, but there are other people being tortured that you can't do anything about, have you not done something good? I don't think this is beside the point - part of the original dispute is to what extent suffering is additive.
Paul, thanks for this.
My first thought was "specks" and after further internal deliberation and reading through the comments, I find myself at the same conclusion. The attempt to compare "specks" with "torture" in this manner is incoherent. It's probably reasonable to assume that practically everyone gets a speck of dust in one or both of their eyes on a daily basis. Dust exists pretty much everywhere humans exist, and the probability of spending your day anywhere outside an industrial cleanroom *without* having dust contact your ocular region is likely very low.
So it's not as if our present position is one of zero knowledge regarding the effect of dust specks in our eyes -- this is something that happens in the real world, every day, and honestly, it's not something that garners a lot of complaint. Torture, however, is superlatively awful *by design*. And I personally would rather live in a world where dust specks were commonplace, but nobody was being tortured, than the reverse.
How a person answers this question probably depends somewhat on how literally they take it. I immediately considered a situation consisting of a comparison between actual dust specks and actual torture, and my analysis was informed by that literalness. However, it seems that some respondents interpreted the question in a more purely abstract sense -- e.g., rather than applying their ethical sense to the real world, they approached the problem as one of getting the "right" answer from a consciously algorithmic standpoint.
I am not getting why some seem tempted to invoke the viewpoint of an entity capable of somehow "feeling" the cumulative effect of tiny amounts of possible badness. If no such entity exists to experience this aggregate suffering, then it doesn't make sense to suggest that imagining an aggregate suffering made up of a zillion teensy annoyances is the best way to reason out one's moral decisions.
Also, the fact that different people are responding to this dilemma in different ways (and with different reasoning paths) hopefully demonstrates that there's more than one way to consider such questions, and that no, there's not likely to be a One True Algorithm that somehow allows a person to make "proper" moral decisions quickly and tidily in every case. A memorized "abstraction machine" cannot be used as a substitute for actual thinking, of the sort that engages directly with the relevant, real, data at hand in a given situation. Sure, abstraction machines can help a person structure their thoughts in some cases, but I firmly believe that people shouldn't let their devotion to keeping their "isms" harmonious override their devotion to the well-being of individuals.
With regard to the original question again, I think the point might have better been served by simply asking whether torturing one person or torturing a whole lot of people was worse. (And I'm sure answers to this would vary as well -- personally I see torturing anyone as just as bad as torturing everyone, but that's beside the point right now).
The "dust specks" example was so inane as to be functionally meaningless in discussion -- you might as well have said, "What would be worse, torturing one person for 50 years, or giving 3^^^3 people a stuffed toy unicorn?" Given that stuffed toy unicorns are bigger than dust specks, and potentially more dangerous (e.g., people could trip over them!), it would seem that there's a greater Existential Unicorn Risk than an Existential Dust Speck Risk. But I'd still choose "Unicorns for All" world over "Torture One Guy" world any day.
Paul/Nick,
I'm still struggling with how tiny dispersed ouches don't *eventually* add up to one big ouch. The only way is if they represent zero harm. Personally, this was my first line of defense for my specks intuition but I decided that the question precluded the harm from actually being zero. In reality, I believe that there is a threshold of suffering below which there is no harm. Consider, perhaps, the level of harm you would be willing to suffer for a 1/N probability (N very large but not too large) that you would save a stranger's life. If you willingly accept harm that has no benefit to you and almost no expected benefit to anyone, can we really call it harm? The specks seem like they would clearly meet this test.
Of course, to rcriii's post, I'm not trying to make this a realistic question. And assuming nonzero harm, it has to aggregate somehow. Nick raises an interesting idea but what would make us believe that the disutility of harm is nonlinear enough to avoid the problem as the number of tiny harms goes to infinity?
Bob, I have to chime in here an say that the original question was _not_ carefully posed:
- Very early in the thread someone noted that 3^^^3 is much greater than the likely number of humans ever. So we are asked to decide between something that has happened in the past and an impossibility.- Elezier did not put any kind of value on the harm caused by the 'speck'. Is it 50/1^^^3 of 1 year of torture, more or less? How can we do any kind of 'calculus' if we don't know?- How do we account for the fact that a speck once washed or brushed out on the eye is quickly forgotten, but torture leaves physical and psychological scars that can last a lifetime? Or are these memorable specks?
Mike, I wasn't claiming equivalence between my question and Eliezer's, only trying to make the point that it's harder to judge under uncertainty if you have incommensurable values.
If 3^^^3 people actually existed, then a 1/3^^^3 probability would not be trivial in absolute terms - for instance, torturing each existing person with probability 1/3^^^3 would be very bad, while it's ridiculously trivial in our world.
Poke 5000 people with one thumbtack each, or poke 1 person 4999 times. I think it's still reasonable to believe that tiny dispersed ouch just doesn't add up to one big ouch.
I agree, but not because they're somehow incommensurable; rather, suffering is linear with single pokes across multiple people but greater than linear when the same person is poked more than once, because of emotional effects on top of the raw pain from the pokes. I would indeed say there is some N between 1 and 5000 where N pokes to 1 person goes from better to worse than 1 poke to 5000 people, but I don't see it as the "magic poke"; it's just the point where a certain increasing nonlinear function crosses a certain constant value. In practice, we don't and can't know the function or the value, so we generally have to decide on principle, and this principle may include discrete "ouchie" and "brutalization" categories; but that doesn't mean they're not both "just" suffering.
Bob: Ok, I'll go along with that. Let's abstract from torture and specks and just say "ouch." Poke 5000 people with one thumbtack each, or poke 1 person 4999 times. I think it's still reasonable to believe that tiny dispersed ouch just doesn't add up to one big ouch. It just doesn't aggregate like that. At some point there's a "magic poke" where you're not just having ouchies, you're being brutalized. And no, I can't say where the number is, but if morality isn't decision theoretic, must I do so?
Nick: To be consistent, I'd have to go with the specking in your case too. But is consistency required? (My intuition about that example is "who cares," because the torture is so uncertain that it can be discounted to zero as long as we don't repeat it.)
Nick:Paul: How, then, do you decide between certain specks and uncertain harm? Would you rather speck 3^^^3 people with probability 1 or torture one person with probability 1/3^^^3?As I mentioned in the other thread, that question and Eliezer's question are profoundly different. In Eliezer's example, you are torturing someone with 100% certainty, at the cost of a trivial harm. (My answer to Eliezer's question is essentially that (trivial harm)*(3^^^3 people)=trivial harm). In you're example, you are choosing a trivial harm against against a trivial (basically=0) probability that someone will get tortured.
3^^^3 people simply cannot be equated with a 1/3^^^3 probability, which, if intelligible at all, is equal to 0 with greater certainty by far than anything else we are "most certain" about in the real world.
"But I truly don't see how anyone can stick with specks under what seems to be the most obvious interpretation of the original question."
The problem here is analogous with the "paradox" of the iterated Prisoners' Dilemma. If the consequences can be completely specified then the problem is one of classical rational decision-making. But in the real-world, any agent is constrained by a point of view that is both (1) located, and (2) limited. For this reason, our best approach to this interesting and realistic class of problems is to implement solutions based on best known principles (and wait to see the actual complex outcome.)
It's like being faced with the problem of a huge chasm that separates one from trading partners on the other side. Does one assume that a bridge is needed, and proceed to allocate physical and intellectual resources to building the best possible bridge? Or does one focus on defining the problem in terms of (1) one's values and (2) best known principles of promoting those values? In the latter case, the process of defining and refining the problem actually leads to defining the solution, which may turn out to be not a bridge per se but perhaps a more highly adapted system of communication and delivery of goods.
Well, I've reached my self-imposed quota for the day, despite this being an interesting topic and one that is crucial to optimizing humanity's path toward an increasingly complex future.
Paul: How, then, do you decide between certain specks and uncertain harm? Would you rather speck 3^^^3 people with probability 1 or torture one person with probability 1/3^^^3?
Paul, but the original question was very carefully posed as different degrees of harm. I share your desire to elevate torture as a different sort of bad. I just can't see how to do it in a consistent way without changing the question. And I think that your (really "our") desire to change the question is one of the more interesting things to come out of the discussion.
"But I'm not arguing that uncalculability comes from subjectivity. While my eventual choice between the resistance or the duty to mother will have to be subjective (that's why it's an expression of character), the problem itself need not have anything inherently subjective about it."
Moral decision-making always entails two fundamental elements: the agent's (1) model of a subjectively desired state, and (2) model of (increasingly) objective principles of effective interaction with 'reality'. Each is crucial to mensuration of matters of morality. Each is differently difficult.
As for the Trolley Problem, it doesn't show that morality is mysterious, but only that the popular conception of morality is flawed. Humans' evaluation of moral issues is encoded at various levels, from innate drives including disgust or pride, to cultural drives expressed by societal/religious norms, to logical reasoning about maximizing expected utility. It's not surprising that differing representations of the problem yield differing solutions, nor that we feel so strongly about questions of what is "right."
Each of these models has in common that they encode, not "what is right" in any objective way, but rather evolutionary "wisdom" of "what works" in principle to further the values of an increasing context of agents over an increasing scope of consequences. We, being a result of these same evolutionary processes, therefore tend to see our branch of "what increasingly works" as isomorphic with "increasingly right."
The practical significance of this is that it takes us a step beyond shallow utilitarianism, expanding our intentional focus from perceived desired consequences in an inherently uncertain future, to *improving the process* that delivers increasingly desirable consequences via increasing effective awareness of elements 1 and 2 above.