Morality Is Overrated

Hanging out with moral philosophers last week at Oxford reminded me of the old complaint that economists neglect morality.  Actually, I think the real problem is the reverse!  Let me explain.

Many people advise us on what to do.  Some discuss personal actions, while others suggest how groups could better coordinate.  And, crucially, some advise us on what we should do, while others advise us on how to get what we want. 

At the personal level, parents, teachers, preachers, and activists tend to tell us what is morally right, while friends, mentors, lawyers, doctors, therapists, and financial planners tend to tell us what will achieve our ends.  At the level of social policy, pundits and wonks give a mixture of rationales for their suggestions.  Moral philosophers, for example, tend to emphasize policies we should pick, while economists tend to emphasize policies to better get us what we want.   

All else equal, we may each prefer to do what is right, but when all else is not equal we often allow other considerations to weigh against morality.  After all, morality is only one of the many ends we pursue.  Yes we want to be moral, but we also want other things, and we each choose as if we often care about those other things more than morality.  (Some say moral beliefs directly cause us to be moral even if we don't want that, but I prefer to describe this as a revealed preference for moral ends, i.e., for "wanting" to be moral.) 

Economic analysis tries to infer what people want, largely from actions, and then tries to suggest policies to get people more of what they want.  (In particular, it suggests good deals - policy packages which should be better for most everyone.)  Yes people often make mistakes, are ignorant, and have conflicts with the wants of others, but economists have many reasonable fixes for such problems.  Critics, however, say economic analysis is untrustworthy because it is incomplete, since wants are only one of many moral considerations.  But this complaint seems to me backwards.

Yes, we "should" (morally) prefer to analyze policy in moral terms, and we should choose what moral analysis recommends.  For example, perhaps we should immediately and drastically cut CO2 emissions because we have no right to pollute natural purity, and we should care greatly about wildlife and distant future generations.  If so, economic analyzes advising only modest CO2 taxes are a moral travesty, reflecting an unconscionable neglect of "non-economic" considerations.  We should thus condemn these analyzes and the economists who support them. 

But in fact we care only moderately about what we "should" do.  We do not want immediate drastic CO2 cuts because we do not in fact care much about natural purity, wildlife, or distant generations, even if we should care more. Economic analyzes suggest modest CO2 taxes not because they ignore "non-economic" considerations - there are no such things - but because such analyzes give morality only as much weight as people do.   

What we humans want is policy that considers our wants overall, without giving excess weight to morality.  So we want policy advisors, like economists, who suggest actions that better get us what we want, even if those actions are immoral.  We do not want to just do what we should, but we instead want to achieve all our ends, including immoral and amoral ends.  So we mostly do not want to just do what moral philosophers suggest. 

Unfortunately, all this is clouded by our tendency to want to appear to care more about morality than we actually do.  We want to take the moral high ground and be seen as supporting highly moral policies, even if we don't actually want those policies implemented.  So we publicly support moral policies when our support seems unlikely to change the outcome.  But it is amoral advisors, like economists, who help us the most. 

Bottom line: We want to get what we want, not just do what we should, and so we want advisors like economists who tell us how to get what we want.  But we'd rather be seen as following advisors like moral philosophers who tell us to do what we should.

Thanks to Nicholas Shackel for stimulating discussions on this. 

Comments

[Hopefully Anonymous]

A dreary and obvious point for your core readers, although pretty well-stated. I'd like more exploration of the politically incorrect policy recommendations it leads to, such as encouraging an emergency room doctor to engage in random murder if the enjoyment of it allows them to be more productive at saving lives.

[Paul Crowley]

Hopefully Anonymous: if you want to show that a consequentialist morality leads to unpalatable decisions, you have to actually demonstrate that there would be some reason for a consequentialist to suppose that the decision would actually have the desirable consequences you set out and that these consequences would outweigh the obvious undesirable consequences. And --- here is the tricky bit --- you have to do that without your audience starting to think that maybe that's not so unpalatable after all.

In this instance I'm pretty unconvinced that the policy you propose would have good consequences, to say the least.

[a. y. mous]

So, if we just want to want what we should want, does to problem go away or will it be merely explained away?

[athmwiji]

Most economists seem to be moral, suggesting that people should respect each other's property and compete fairly. I think very few economist would advise someone to get rich through conquest and oppression by force of violence.

[Roko]

"We want to get what we want, not just do what we should, and so we want advisors like economists who tell us how to get what we want. But we'd rather be seen as following advisors like moral philosophers who tell us to do what we should."

- I think you're under-rating the importance of ethics here, and also slightly misinterpreting what it actually is. You talk about ethics as if it's just another preference that humans have, or even as if it's a commodity, but I think that this is not the case. If people preferred to act morally, then we wouldn't need to talk about "ethics" or "morality".

The point is that a lot of the time, people like to mess things up for other people, for example by killing or hurting them. Ethics is a mutually-enforced contract within a society which makes people behave in a way that they don't actually want to. Criminals would love to be able to get away with robbing people's houses - they have a preference for getting money without doing any work - but using our codes of ethics, and the enforcement instruments thereof, we stop them.

Ethics almost always boils down to the need to resolve some "tradgedy of the commons" scenario. In the case of the robbers, it would be better for the individual robber to rob people's houses, but it's worse for everyone if everyone does it.

Should we provide criminals with expert advisors who will help them to more effectively commit crimes? No. Because what they want is not for the common good. In fact, we should probably outlaw such advisors.

[a. y. mous]

>> No. Because what they want is not for the common good.

Roko, the common good is the correct choice, "by definition". Right? Forgive me. You are right. That was a cheap rhetorical retort at the OP.

I've been reading posts here at OB for a while now. Have begun commenting only recently. Over-arching and over-reaching reductionist analyses makes me want to clarify then and there that it ain't so.

[Barry Kelly]

I'm afraid you lost me a bit on this one. Economics is the study of choices and how individuals maximize utility and firms profits using those choices. It doesn't have a moral dimension at all.

Even if an economic model which has a wide range of choices for the individual leads to outcomes that a moral philosopher might object to, that doesn't amount to a moral consequence or conclusion of economics: it's a consequence or observed morality of individual choices.

To achieve a desired moral outcome, an economist might recommend limiting certain choices, adjusting incentives and otherwise tinkering with the rules of the game and the information transmitted by price, but that morality is imposed from without.

In the case of environmental harmful effects, it's pretty basic economics that the costs to the group are not generally borne by the instigator of the effects, and that's why the harmful effects continue. You can't draw a conclusion that because the market resulted in harm to the environment, that represents a moral choice of a collective body of people, even if it was caused by the individuals making up the body. Instead, it comes down to basic game theory and the tragedy of the commons - which, incidentally, only occurs in large impersonal markets, not in the original villages and farming communities where people had a small enough circle to remember and punish infractions against the group.

Thus, for the specific example of environmental harm, one needs an external value judgement to e.g. implement Pigouvian taxes to internalize these external moral costs into the pricing information.

[Robin Hanson]

Hopefully and Paul, this is not an apology for consequential ethics - it is an apology for ignoring ethics and just looking for deals that get people what they want.

a y, if you have conflicting wants, such as wanting to want something other than you want, we may prefer to observe your behavior to see which wants are stronger.

Roko, moral philosophers will tell you morality is far more than just a way we coordinate to punish people who prevent us from getting what we want. People trying to get what they want can want to punish criminals without reference to morality.

Barry, the issue is the size of the tax, not whether there should be a tax. The tax to correct harms to wants seems much smaller than the tax to avoid immoral outcomes.

[Barry Kelly]

Re "issue is the size of the tax": the rules of the game are pretty simple here too, IMHO. What do you mean by "correct harms" vs. "avoid immoral outcomes"? If by this dichotomy you mean minimal and maximal moral shaping of behaviour, then I think you're ignoring the obvious: a maximal moral shaping is inefficient because it incurs other moral costs from other sources. For example, there may be a moral cost to impinging liberty. Having a maximal tax to "avoid immoral outcomes" in the specific context of e.g. CO2 would have other costs in terms of liberty. Optimizing in the context of *all* the moral costs (which, to reiterate, are external to economics) should tend towards minimal shaping, *assuming* e.g. the Western individualist approach to morality.

[Phil Goetz]

You're saying that "morality" is a way of getting things that you "should" want; "economics" is how to get things you actually want.

Let's eliminate the distinction between what you want and what you "should" want. At root, we do want a clean environment; we just want other people to want it more than we do. The term "should" indicates a collective want reified and glorified to sucker other people into pursuing it more vigorously than is rational.

Here are the definitions that I think underlie Robin's post. (I'm using these terms in an artificial way, for the purpose of discussion, so don't thing that the things I say about "moral values" here are things that I believe about "moral values" as we usually use the term.)

Values: What you/society want. Food, happiness, security, etc. Evolved into people.

Economics: How society can achieve its values.

Moral values: A second layer of values that encourages people to act in ways that tend to attain society's primary values. Courage, honesty, sociability, etc. Also evolved into people, though not as consistently. It is easier to defect from moral values than from values.

Morality: A set of moral values, bundled with heuristics for how attain them for people who don't understand economics.

Robin is additionally defining morality as ignorant of economics. This is what we observe, but it's a historical accident, due to the fact that we live in the time when basic economic principles have been developed, but not long ago enough to have been incorporated into mainstream moralities.

I think the distinction between morality and economics is not one of rationality. I would rather define morality as: How society can achieve its moral values.

A modern, well-informed morality would insist on using economics to achieve its moral values. Our moralities do gradually improve as we gain economic understanding. For instance, early Christianity was radically communist. It shed this gradually over the centuries, eventually allowing ownership of property, charging of interest, enforcement of debts, and respect for wealth.

Many people would disagree, and say that morality means doing "the right thing" yourself, without regard for what other people will do, because you're only responsible for your own behavior. E.g., give money to the homeless person on the street even if you know they're going to use it to get drunk, because you have fulfilled your moral duty, and it's not your fault what she does next. This is a greedy morality, in 2 senses of the word: It's algorithmically greedy because it tries to jump directly to attaining your moral values without having to think; and it's economically greedy, because the goal of such a morality is not to attain society's values, but only to attain your own personal salvation.

Anyway. If there is a distinction between economics and morality, I would say that it is one of these:

1. Economics teaches us how to attain values; morality teaches us how to attain moral values.

2. Everyone desires values to roughly the same degree, whereas the desire people have for moral values varies greatly. Also, moral values differ from values in that it is generally advantageous to you to pursue values, and to have other people pursue moral values. This means that morality has a game-theoretic dimension beyond what economics does. When one agent considers what public morality to support, he has a trade-off between the cost to him of behaving in a way that appears to adhere to that morality, and the benefit to him from other people - who differ from him in the enjoyment they get from moral values, and in the degree to which they can be suckered into sticking to those values beyond what is rational - following that morality. When people set forth moralities in public, the equilibrium morality is the one that, over the population as a whole, may attain optimal returns; the trick is that some people are receiving their payoff in good feelings and self-satisfaction, or expected payoff in the next world.

We thus predict that the more irrational the population is, and the larger the percentage of defectors is, the stricter the equilibrium morality will be.

This suggests that morality as we know it may be /possible/ only in the presence of a large number of defectors. If everyone were honest, there would be no advantage in promulgating social standards stricter than you want to live up to yourself.

Postscript: When putting forth a morality, the game-theoretic, rational behavior may be to deny the validity of game-theoretic, rational behavior.

[conchis]

As an argument that economists shouldn't take morality into account this seems hopelessly self-defeating. You seem to be assuming that economists should have some sort of normative commitment to giving advice that gets people what they want.* Now, either:

(1) you have some sort of moral argument for that position, in which case you might not want to diss morality so much; or

(2) your only available response when I say "but I don't want economists to just tell people how to get what they want; I want them to take morality into account" is to tell me that you don't want that. At which point I'll shrug, and wander off.

*I guess you could be making the argument that people are more likely to listen to advice that conforms to what they want. But that doesn't seem to be the argument you're making, and even if it were true, I don't see that it should make me care any more than I already do about feasibility concerns.

[conchis]

On rereading, maybe I mistook the point you were trying to make. If so, apologies. I'll stand by the substance of my comment, but perhaps it's not actually disagreeing with anything you said.

[Unnamed]

(In particular, [economic analysis] suggests good deals - policy packages which should be better for most everyone.)

This parenthetical may be the most important sentence in Robin's post. The question is: who counts? Who is part of that "everyone?"

If a potential U.S. policy benefits most Americans but harms most of the rest of the world, what recommendation will the economist give? What if it helps most people who are currently living to get more of what they want, but makes future generations of people get less of what they want (and more of what they don't want)? What if it mostly benefits humans, but mostly harms non-human animals? What if it benefits voters at the expense of non-voters, adults at the expense of children, or born humans at the expense of human fetuses?

These questions do not arise for the other "amoral" advisers, since a doctor, lawyer, or financial planner is advising one individual about what will achieve his or her ends. Economists, to a large extent, are advising individuals about how to get others more of what they want. So the question naturally arises, "which others?" (Another question arises as well, about how to consider tradeoffs between those others, but I won't get into that here.)

[michael vassar]

I'm skeptical of the claim that everyone desires values to roughly the same degree. What about depressed people for instance, or various types of ascetics?

[Phil Goetz]

Given what I said above about morality helping society attain its moral values, couldn't you argue that morality ought to be legislated?

Perhaps our strong experience indicating that morality ought not to be legislated, is only because the moralities we are most familiar with were designed in ignorance of economics, and were refined by defectors to exploit irrationality rather than by society to use force of law.

Mike: I think people's desires for values are more similar than their desires for moral values. Both vary. But you won't find many people who don't want food and water, and I think you won't find as many ascetics as amoralists.

I mentioned it because if people vary in their preferences for moral values, then there is more incentive for "trade" between people who get pleasure from behaving morally, and people who don't.

I shouldn't even use the word "pleasure", since we're also discussing people who don't value pleasure. "Positive feedback".

It's funny that we don't have a word for the most fundamental drive in human behavior.

[Richard]

Robin, can you clarify what normative claims you are making here? Your title makes the evaluative claim that "morality is overrated", and in your comment you say you are offering "an apology for ignoring ethics and just looking for deals that get people what they want." But I don't see that you've offered any argument for these normative claims.

Your post is full of merely descriptive claims -- your "bottom line" merely describes what people want. It's not obvious that anything interesting follows from this, unless you also care to argue that 'what people want' has normative relevance - is *worth* satisfying - even when it's contrary to moral requirements.

Phil Goetz - you appear to assume a merely instrumental conception of rationality. Note that many philosophers would reject this, and indeed hold that it at least cannot be irrational to do as morality requires (and some would further claim that it is rationally required).

[Phil Goetz]

Phil Goetz - you appear to assume a merely instrumental conception of rationality. Note that many philosophers would reject this, and indeed hold that it at least cannot be irrational to do as morality requires (and some would further claim that it is rationally required).

Thanks for the link; otherwise, I wouldn't have had a clue what you meant.

The idea that you can rationally criticize values according to their consistency is very interesting. The example of someone who cares what happens to them until New Year's Eve, but not afterwards, is good.

But your larger example - which I think is your motivation for holding the position - that, rationally, we should extend our consideration beyond our immediate friends and family - is dead wrong. The instinct to value friends and family has been partly explained by evolutionary psychology, and experimentally validated enough that we can say that's probably the right explanation.

This is why it is easiest to treat values as arbitrary. Our values are evolved instincts, and evolution can create arbitrary values. The evolved values are the ones we have to work with. If you start critiquing them rationally, you'll end up with abstract philosophical values that no one can live with, and that aren't evolutionarily stable. You'd be repeating, in a way, the moral mistakes of religion and Marxism.

And you'd have a serious onion-peeling problem trying to draw a line between aspects of values that are irrational, and the root values themselves, which are by definition irrational.

[Richard]

The question whether our values are coherent (rationally justified) is logically independent of the question how we came to have them. To think otherwise is to commit the genetic fallacy.

Granted, human psychological constraints are something that needs to be taken into consideration by moral philosophers. It's one thing to work out what the ideally rational agent would value, and quite another to specify what imperfect beings like ourselves ought to value. Roughly: we ought to adopt whatever 'practical morality' would in fact cause us (given our actual condition) to come closest to realizing the ideally rational ends.

But that's no excuse at all to simply ignore the rational question and treat all possible values as equally arbitrary. (Certainly not in an academic forum like this, where we are presumably trying to get at the truth.)

[Phil Goetz]

Richard: The idea that you can rationally decide what values to have, rather than taking the values that have been constructed by evolution, is even less feasible than the idea that you can rationally decide via inspection what the symbols in a knowledge representation "mean", without taking into account how those symbols are grounded in the environment. In the latter, you can at least test different hypothesis for internal consistency. Values are not required to be consistent; in fact, they always come into conflict with each other.

I am not aware of any way of rationally choosing values. I don't think the ideally rational agent would value anything. If you construct a large, complex, rational artificial intelligence with lots of knowledge, it won't do anything unless you assign it values/goals.

What is the genetic fallacy?

[Eneasz]

Have you looked into Desire Utilitarianism? It seems like it would address most of your concerns. As far as I've been able to find, it's the ethical theory that best explains current actions and best predicts future actions. It also easily refutes the notion that 3^^^3 dust specks is worse than 50 years of torture without all the mental contortions that act utilitarians have to go through.

[tobbic]

It seems to me that it would be beneficial for someone to strictly define the concepts in the discussion (morality, want, etc.).

I agree with Robin Hanson that values (which constitute a morality) are essentially same as wants (if I understood correctly). Both express preference but on from different perspectives.

Maybe morality and the consequent values represent a penalty function whereas wants represent an utility function. It seems to me they can be superimposed (and thus are the same thing). I think in the case of many ppl the penalty function is not a step function getting '-inf' after some fixed value has been violated. However, my impression is that ppl tend/like to think values as absolutes rather than wants that can be traded off.

[Eliezer Yudkowsky]

Phil, you had 7 of the 10 most Recent Comments in the sidebar. This should never ever happen. 2 is the usual limit, 3 in very rare circumstances, never 4.

I've compressed some of your comments together to cut it down, but still, wait to post your replies until you see fewer instances of your name in the sidebar.

Also, your comments were very long and you should consider writing your own blog post, then linking to it here.

[Unknown]

Suppose you could take a pill which would instantly make it impossible for you ever to desire to do something which you believed to be immoral (unless you changed your mind about it first, i.e. completely before starting to desire to do it). Would you take the pill?

Some people would take it. I presume that Eliezer would take it, based on earlier things he has said on similar topics. I would definitely take it.

For someone who would take it, it doesn't make a lot of sense to say that his desire not to desire immoral things is weaker than his immoral desires, even if in practice the latter desires are found to guide more of his actions than the former desire.

[Eliezer Yudkowsky]

Robin, I do not think you are using the word Hanson::morality the way I use the word Eliezer::morality. What you call "morality" I call "moralizing" or "sanctimony" or "self-righteousness".

Unknown: Suppose you could take a pill which would instantly make it impossible for you ever to desire to do something which you believed to be immoral... I presume that Eliezer would take it

If so, I would be doing it either for instrumental reasons (makes it easier to help others) or weakness of will. As a strictly local act, taking that pill would be immoral.

My reasons for saying this should become more apparent once I get into the sequence on Fun Theory (no, it doesn't start today) but roughly, if you simplify your psychology past a certain point, you stop being an end, and end up as a means.

[Richard]

Phil - of course, we reason (practically) from the values we start with, just as we reason (theoretically) from the beliefs that we start with. In both cases, rational pressure induces us to resolve inconsistencies, minimize arbitrariness or ad hoc distinctions, and build on our existing attitude (belief/desire) sets so as to make them more internally coherent, unified, etc. (That's why the post I linked to argues from our existing values - concern for ourselves, friends and family - to a more coherent value set that also includes concern for others who are relevantly similar.)

N.B. You seem to be assuming some sort of foundationalism, according to which rationality starts from scratch, deriving conclusions from "self-evident" first principles. I'd agree that's a hopeless project. Fortunately, there's an alternative: coherentism. Start with what you've got, and ask whether there are any ways it could be improved. Frankly, I'd be stunned if the answer were 'no'.

"What is the genetic fallacy?"

It's a fallacy of irrelevance, like ad hominems, whereby you look exclusively at the historical origin of an attitude rather than assessing it on its merits. (Example: "People believe in God because it makes them feel good. Therefore, God doesn't exist.") Essentially, it is to confuse the projects of *explanation* and *justification*. To explain a belief or value is not thereby to show that it is wrong or unjustified. (It might be wrong or unjustified, but that requires further argument -- philosophy, not psychology.)

[Unknown]

Actually, after thinking about it a bit more, I would have to add some conditions in order to take the pill myself. But I still don't see any reason why taking it would be immoral, so I'll just have to wait for the Fun part...

[Constant]

This should never ever happen.

If the concern is that other comments are pushed away, you can vastly improve the user experience of those not wanting to miss comments by adding a "more..." link at the end of the "recent comments" pane and a separate page with a longer list of recent comments. Of course, if your purpose is to concentrate the mind by setting up arbitrary rules, you could require all commenters to post in haiku format.

[Phil Goetz]

Desire Utilitarianism (googling) is Preference Utilitarianism with energy minimization to find a set of preference weights that gives a local maximum of preference satisfaction. It sounded interesting at first, but a few minutes' reflection shows that it leads to either Buddhism or Satanism.

[Michael G.]

How do we make it so that what we want is what we should do?

[Eneasz]

Phil, could you expand on that, if you have the time? I am flabbergasted as to why you'd think it would lead to either Buddhism or Satanism.
For two quick reference points (well, not that quick, but not too long), the creator of DU keeps a blog, which often delves into the theory. Much of the basic groundwork is covered in:
http://atheistethicist.blogspot.com/2007/12/morality-from-ground-up.html
and
http://atheistethicist.blogspot.com/2008/01/kmesons-question-acts-and-desires.html

Michael, I'd point you to http://atheistethicist.blogspot.com/2008/01/good-resolutions-and-how-to-keep-them.html although I fear at this point I'm begining to look like a bit of a fanatical disciple. :)

[TGGP]

Economist can do research in order to learn about what is efficient. There is no productive research in morality because it is an empty field, like theology or astrology. It has no impact on anticipated experience. Seeing as how there are no moral facts, there is no reason for them to be taken into account.

[TGGP]

Richard's talk of coherent justification reminds me of traditional rationality as social rules.

[poke]

Morality is just bad psychology. Normativity is one way a descriptive theory can fail.

[Paul Gowder]

Robin,

I don't think you're saying what you think you're saying.

Or, to put it differently, "Overrated" with respect to what?

You seem to be saying that morality is overrated with respect to what we want, i.e. we claim morality is more important to us than it is, so we overrate (in our utterances) the true value (as a matter of desire satisfaction) of morality.

If that's true (and it probably is), then you've stated a descriptive fact about the world. We claim to care more about morality than we actually care about it. That's fine and dandy.

But you want to push it further. You say that it's a "problem." And you say that your post constitutes an apology (a justification) for economists not focusing on ethics.

How could it be such a thing? I'll invoke Hume's Law here. The claim as I've stated it has only descriptive premises. It can't constitute a claim that morality is overrated, or that it's OK for economists to focus only on what we want.

Unless... unless... there's a normative claim under the hood! Something... something about what we ought to do, given our preferences... And what might that normative claim be?

[Robin Hanson]

Phil and Eliezer, I'm trying to use the word "morality" the way moral philosophers use it.

Conchis, I'm saying your wants are rare - few want that.

Unnamed, I mean to include all nations, all time, and even all species.

Michael V, I don't follow you.

Richard and Paul, I'm not making "normative" claims, just claims about what we want. Call them "merely descriptive" if you want - I and many others find such claims persuasive regarding the actions we will take.

Eneasz, you miss the point I think - I'm not looking for moral views.

Tobbic, I'm not at all saying values are the same as wants.

Michael G, I don't want to do that at all.

[Unnamed]

Robin, do you mean that you're including what future people want among the "wants" that are the subject of purely economic advice? How does that square with what you say about CO2 emissions, where you seem to be counting future generations as a "moral" consideration?

[billswift]

Susan Haack's Evidence and Inquiry develops a new basis for epistemology by bringing together foundationalism and coherentism in a fashion where each part strengthens the weaknesses of the other. She hasn't been completely successful, but it is a useful idea and a good start.

For a more rigorous basis of ethics, you might try Merrill's The Ideas of Ayn Rand. He suggests (pages 106 through 109, of the Open Court paperback edition) that oughts can be derived from facts "So, if we can agree on what morality is to accomplish, we can develop moral rules from factual statements." Basically, he advocates unifying the operational use of ought with its normative use. Where operational use is of the form "If you want to accomplish this, then you ought to do that." Again like Haack's ideas, a good start though problems remain.

[billswift]

Sorry, I meant to include a couple of sentences on Haack's theory, which she referred to as "foundherentism". Rather than axiomatic foundational beliefs or beliefs that are strictly judged by their coherency with other beliefs, the theory she developed used a relatively small set of foundational beliefs which were themselves supported by their mutual coherency. I can't really describe it in any more detail since I haven't finished reading the book.

[Robin Hanson]

Unnamed, the weight a group would get in a deal need not be the same as the weight it morally should be given. Distant future generations get little weight in deals made today.

[Richard Hollerith]

Eliezer writes, if you [do X], you stop being an end, and end up as a means.

Aw, come on. Jump in. The water's fine.

[frelkins]

I understand Robin to say that he doesn't care much about morality and neither do most people, despite their protestations. And that's cool. But what I think may be important to note is that a market actually can - or could - take most people's rather skimpy 'tude towards morals and yet produce the most beautiful and moral ends. One of the benefits of capitalism is that it does not require a pure heart to enable its actors to en masse do good.

[Phil Goetz]

Eneasz: Re. Desire utilitarianism:
Let the space X represent your possible actions. You have a set of functions f_i(x), for x in X, giving the degree to which action x satisfies desire i. You have a set of weights, w_i. Utility for action x is the sum U, over all i, of w_i*f_i(x). (Sum over all desires for all individuals. Note that each f_i accounts for all future repercussions of your action, appropriately discounted for time.)

Desire utilitarianism adjusts the w_i to maximize U. An example given was that 1000 people want to torture an innocent child. U is highest if you let them do so. But dU/d(w_t), the rate at which U changes as you change the weight w_t on their desire to torture children, is negative, because increasing that desire in people leads to more frustration of other desires. Hence, you should decrease w_t.

But then you have to decrease w_t for 1000 people. It's easier to increase w_u, the weight assigned to the desire to be tortured, for the innocent child. (And if you can't change weights arbitrarily, and make people enjoy being tortured, then the whole program of DU falls apart.)

We like having a desire fulfilled, and dislike having it frustrated. So every function f_i will probably be monotonically increasing along some dimension from < 0 to > 0, then declining after saturation. This means that for every set of weights w_i that provides a local optimum at x, the set of weights -w_i probably provides a local optimum, maybe around -x. That's the "Satanist solution", in which all our desires are inverted: People enjoy torturing each other, and being tortured.

The Satanist solution probably gives a higher U, because it's easier to fail than to succeed. You will achieve a higher U if you persuade everyone to enjoy pain, failure, and loss, and then set about wreaking havoc.

Buddhism is the trivial solution in which you set all w_i = 0. It is an optimum if satisfying desires is zero-sum. It is the optimum when satisfying desires is less-than-zero-sum. It isn't clear that it isn't always the optimum.

[Unnamed]

Distant future generations get little weight in deals made today.

So your example of only modest CO2 cuts is a policy that harms lots and lots of people who have low weight (distant future generations), not a policy that's "better for most everyone." Does that mean that you're just doing standard utility calculations, summing (over individuals) weight times impact?

How are these weights set? Giving some individuals more weight than others sounds like a moral stance to me, and not just value-neutral practical advice.

[Psychohistorian]

"Economic analysis tries to infer what people want."
Pretty sure this is where problems started.

Any attempt to infer what people want is, in this context, moral. Do we give everyone equal weight? Do we give the poor more weight? Do we give the rich more weight? Do we give the more informed more weight? Etc.

More to the point, I don't see how an economic analysis is able to conclude what people want. An economic analysis would, for example, create function expressing y (carbon output) in terms of x (the amount of a carbon tax). Nothing in economic analysis can tell you the appropriate target value of x. Instead, we have some system for determining an appropriate value of x. In the case of the US, x is determined via representative democracy.

RH would probably say that x (or y) should be determined by a prediction market. This itself is something of a moral value. Another person could say that the tax should be set by those it will most harm, or those it will most benefit, or those who can best determine its effects.

Economics is not objective when it comes to finding what people want. It is a very specific system that assigns different weights to different people based on complex criteria. Claiming that what one determines via economics is "truly" what people want is claiming far too much objectivity for the science.

[Z. M. Davis]

Constant, I should think
That that kind of snarkiness
Is unbecoming

[Richard Hollerith]

Z. M. Davis reads
The Logic of Science by Jaynes,
Stops to write haiku.

[Constant]

It is not a snark
If the person who wrote it
Thinks it sound advice.

[Michael F]

Yes and No. If I want something and it causes harm to someone else in my getting it, it is immoral. However, lets assume that I consider the harm done irrelevant as long as i get what i want. Now let's say, another has the same want and it causes harm to me in their acquiring it. Again, assuming that it is irrelevant to them what harm they caused me in their acquisition of their want. In the first case, morality gets in my way; in the second case morality is paramount. Since the actual case is billions of people seeking to fulfill their wants, some governing morality will assert itself. It is reflected in laws and it is true that some seek to get what they want by finding was through the maze of the human contract regardless of the damages done. It is true that economists seeking the the economical path to the acquisition would take morality into account, all thought they do if they work within the legal construct. It is not likely an economist will tell you to just steal what you want from your neighbor. Although, a case could be made that war over resources, such as oil may very well be just such a suggestion. Although, there is still a moral reasoning made. To act completely without regard to morals is what we call criminal. What a slaughter house we would live in in that case. The golden rule did not come into existence by accident.

It is not that we want to be seen as following advisers that tell us what we should, it is imperative that we do so else we risk losing all to another at any moment. We would need to be prepared to defend our ownership of our acquisitions at a primal level, for we would certainly never have advanced this far had pure economic considerations been at the heart of our motivations. We are humans and no man is an island. Humans have many wants and some include intangible items such as companionship, reproduction, and enjoyment. Morals are not a fancy invention of some wacky philosophers. They intrinsic to the human dilemma and survival of the species.

Things are not equal and circumstances are not all fair. To strive towards making all things equal and fair without idealizing it but rather considering the human condition, competition and honest gamesmanship without cheating is not perfect but morality does guide it in whatever state it maybe. Even economist would not suggest just stealing it from your neighbor.

We can not want to stop all CO2 emissions now because of the jobs dependent upon them. These people would be harmed. Thus we look to ways to remedy the situation as it is. Where exactly does morality become ridiculous and idealized is hard to say but even an economist functions within legal parameters and do attempt to make decisions based in deeply moral opinions.

Well, so it seems to me.

[tobbic]

Robin Hanson,
I am interested to know what is your argument that it's not fruitful to considers values and wants as equivalent in some contexts.

Man is a moral animal and that separates us from other species. I think values are largely a manifestation of social norms and empathy.

Basically there i'd say is a spectrum of levels in which a thing could be desirable/undesirable:

Value/want 1:
1) "I believe that premarital sex is wrong/ I don't want to do it"... "because it causes me great physical pain." (individual context)
2) "I believe that premarital sex is wrong/ I don't want to do it"... "because i'm afraid that there is a chance that others in our community will find out and diss me for the rest of my life which is horrible because i derive much of my joy (individual context) from social interactions" (social context)
3) "I believe that premarital sex is wrong/ I don't want to do it"... "because my belief system is based on some form of catholic doctrine and it would cause me either a) great cost in the form of having to restructure my pretty well working belief system (I still believe in some form of logic & consistency) b) a great cost in the form of having to self-deceive" (introspective context)
... (relevance of considerations is quickly decreasing)

If 1) is a desire/want, why is not 2) or 3) a desire or want?

[Robin Hanson]

Unnamed and Psycho, I can't explain standard economic analysis in detail a short blog comment.

Michael F, morality is not the only or best way to deal with conflicts between individual wants.