57 Comments

Carl, on longer timescales, see this post.

Expand full comment

All of Robin’s original concerns are about long term effects on massive future populations

Elsewhere in this thread Robin talks about focusing on humanlike ems in era measured in "doublings" i.e. a period of a handful of years or less (with Robin's current growth rate estimates) out of trillions, concentrated in the Solar System rather than the accessible universe. From a total utilitarian point of view that's pretty negligible.

Expand full comment

Carl, at least ten doublings.

So old-style humans can survive on the margins of an em society for at least 20 weeks (with your estimates of doubling time from elsewhere)? Great news, that!

Expand full comment

" There’s the quality of the lives of others, which Robin and some commenters express concern for but you don’t care about. "

Posturing as pro creating ems because you are confident of some quality of life they'll experience is different than expressing concern for the quality of lives of others.

I'd contrast earthy survivalist concepts of concern for others with the "flood the market with ems" concept of concern for others.

Expand full comment

Hopefully Anonymous, it seems there's a couple of issues we need to distinguish. There's the quality of the lives of others, which Robin and some commenters express concern for but you don't care about. There is the question of whether your consciousness will persist if you are uploaded, which you have earlier expressed concern about. And finally, assuming your consciousness does persist, there is the question of what your quality of life will be like. So the logical chain of inference is really only relevant for that last question.Also, are you saying that you believe Hanson to be similarly egoistic and lacking in a sort of kin-altruism for partial descendants?

Expand full comment

@Robin

I count gains to potential creatures as well as to current existing creatures.I can understand that to some extent. If you don't want people to have unsatisfied preferences today you wouldn't want them to in the future, even if they don't exist yet. But I don't see any inherent moral problem with people in the present taking steps to shape the future so that the types of potential creatures capable of existing in the future are limited to types that satisfy present-day preferences to some extent. Future creatures don't have preferences yet since they don't exist yet so the ones that will end up not existing will not have their preferences violated. I see no moral problem with attempting to restrict the population of future creatures to ensure a higher average level of preference satisfaction.

I do not think ems would quickly “eat” humans for their natural resources.Knowing that humans would be eaten slowly instead of quickly isn't much comfort. And what about other, less efficient ems? I wouldn't want them to be eaten either.

I'd be less bothered by this scenario if I thought that every em clan that was created would survive and live decent, moderately wealthy lives ("clan" is the word you've been using to refer to a series of ems copied from the same uploaded mind, right?). When I read the novel "Kiln People" I didn't consider that to be a horrible future (on the contrary, I thought it was awesome!), even though it contained many ems who slaved for their entire short lives at work. That was because, even though individual ems were poor workaholics, the em "clans" were wealthy and managed to find plenty of time for fun. And since the "clans" shared all their memories, each em got to share in the experience of having fun and being wealthy, even if it was another em that was doing that while they were working. If the em scenario you described sounded more like "Kiln People" I probably wouldn't object to it nearly as strenuously.

That's not the impression you've given me, however. It seems like you think that ems will constantly be working and constantly be poor. And that some entire clans of ems will be "eaten" by other ems that are more efficient at being workaholics. That doesn't sound like a future worth creating to me.

Also, I want to address your statement that:

I predict we will not coordinate to prevent a great population increase and wage fall.Isn't this a situation where a simple minimum wage law would do the trick? In the modern world the minimum wage is cruel because it restricts the amount of jobs available, preventing already-existing workers from getting jobs. However, it seems to me that a simple job-destroying minimum wage law during an em explosion could easily limit em population. I doubt anyone will go to all the trouble and expense of running off a new em unless they think they have a job lined up for them. In this world the minimum wage law, instead of throwing already-existing people out of work, it would simply prevent new people from being created.

Of course, it may be that such a policy, if not adopted universally, would result in some areas with no such laws becoming hugely populated with ems. But if every country in the world managed to sign and enforce some kind of treaty or something, would it work?

Also, on a somewhat related note, I thought I'd semiseriously suggest that sometime you write about the sort of society in portrayed in the classic cartoon "The Flintstones." While that world is obviously economically and sociologically improbable, it has a feature that you might like, almost all household appliances, vehicles, and machines, are sentient creatures capable of having preferences and enjoying life. Would you consider that society superior to ours, with its cold, emotionless household appliances? It seems like a fruitful avenue for moral analysis on your part, and would probably make a really fun post. Just a friendly suggestion.

@Anonymous

I think your points 3 and 4 against em happiness can be resolved by mind (re)designThat sounds utterly horrifying. Totally rewriting an innocent person's values like that is not ethical. To make an analogy, imagine if the plantation owners in the antebellum South had discovered a foolproof brainwashing technology for their slaves.

Expand full comment

What basis is there for thinking there will ever be a “post scarcity society”?

As long as status has some value, status will be zero sum. If status is zero sum (which it is and inherently must be) then there will be motivation for those with less status to make other things zero sum (i.e. cause artificial scarcity) so as to use value in one zero sum system to trade for status (which is and can only be zero sum). That is one of the main uses of wealth now, to acquire status. Wealth is only useful to acquire status to the extent that wealth is scarce.

If democratic governments control resources, then whoever controls the government controls those resources. How much profit did Halliburton make from the Iraq war? On 01/10/2002 Halliburton stock was trading at 5.77. On 4/13/2006 it was 38.56. Were all of those no-bid contracts that the US government let to Halliburton contracts that Halliburton lost money on?

Why are we not in a “post-starvation society”? There is plenty enough food to feed everyone, but shortages in food and food delivery are used by some to gain other things. North Korea is negotiating for food aid to prevent its people from starving. Why is there famine in North Korea? Because the government in North Korea values retaining power more than it values the North Korean people not starving.

Why is the US not a “post-no-health insurance society”? Many other countries are, and they spend less per capita on health care than the US does.

Expand full comment

"of sick" was a typo - has been corrected in the comment.

Expand full comment

"If those few were allowed to copy themselves, that would produce subsistence wages for such tasks."

I was specifically imagining that there would be no wages at all, that's why I said "volunteer"--it would be the equivalent of editing wikipedia for fun. Again, in a post-scarcity economy it seems plausible that something like a comfortable guaranteed minimum income would be seen as a default right, just as health care and education are considered default rights in western welfare states. I'd think quite a lot of people would spend time volunteering any in-demand skills they had if they they had no pressing need to work for a living. Do you have some economic argument to think the creation of a guaranteed minimum income would be unlikely in a post-scarcity society where all the work that's necessary to keep society running (particularly an upload society, whose only real necessities are the manufacturing, maintaining, and powering of lots of computers) can be done in a fully automated way by non-sentient AIs?

Especially of sick folks “see copies that haven’t diverged too much as fairly disposable.”

Why do you call it "sick"? I'm not suggesting that copies would be involuntarily deleted, screaming, against their will, just that it would become a common attitude to have no problem with creating a bunch of copies of oneself and planning in advance that all but one will be deleted after some task is completed (with no copy knowing in advance that it will be one of the ones deleted, as opposed to being the lone survivor). If one believes in something like the theory of quantum immortality, one should have no problem with this, because one will always experience being the copy that avoids deletion. And for anyone who believes in quantum immortality, the main ethical argument against Tegmark's quantum suicide experiment (discussed in the section of his site "The Interpretation of Quantum Mechanics: Many Worlds or Many Words?", about 1/4 of the way down the page here) is that it would cause suffering for your friends and family in the worlds where you died (even if you never experienced these worlds), but this argument wouldn't apply to the case of creating a bunch of copies and then deleting all but one.

The point I'm arguing here is that regardless of whether quantum immortality is "true" on a metaphysical level, I think it's an idea that would tend to become mainstream in a society of uploads, for two major reasons:

1) Those that are willing to regularly run these sorts of Darwinian experiments on themselves would likely be more "successful" in many ways, particularly in the case of trying to alter their own brains to increase their intelligence, but also just in the sense of there being a lot of demand for their skills by others (if you want to use some of the bits you own to host an upload who will perform some useful task for you, would you rather temporarily rent out those bits to an upload who will delete themselves after a time, with the bits then reverting back to you, or would you rather permanently spend them to create a new copy who will need them indefinitely to keep running, and who will possibly gain dangerous political power if there is a lot of demand for copies of that individual so lots of people are donating their bits to make more of them?)

2) Even if an upload might feel anxiety about the prospect of creating a lot of copies and deleting the majority the first few times, after multiple rounds the belief in quantum immortality would become very natural and intuitive, regardless of whether it's metaphysically "true", since the copies remaining after several rounds would naturally be those who had memories of repeatedly finding themselves to be a survivor in each round. As an analogy, all of us biological humans find it natural and intuitive to feel some sort of persistence of identity over time, feeling that we are the "same person" as the version of us who existed a few years ago, even if we know that all the atoms making up our current brain are different from the ones that made up "our" brains a few years ago, and even if we might find it intellectually reasonable to take the metaphysical position that persistence of identity is an illusion. Our memories are too convincing on a gut level for us to really act as though we believe it's an illusion! Similarly, although people today often question whether their consciousness would really survive a destructive mind uploading procedure or whether "they" would die and the upload would simply be a copy with false memories, the upload itself would no doubt find its memories to be just as convincing as we find ours, and on a gut level would feel just as much like the same person regardless of what it thought about the issue intellectually. What I'm saying is that this sort of gut-level belief in something like quantum immortality would be similarly convincing for any upload that had survived multiple rounds of copying-and-deletion, so there would be a sort of selection process in favor of uploads that at least act as though the belief were true.

So, as a thought-experiment, imagine if you were an upload who found the notion of quantum immortality every bit as believable as the idea of persistence-of-identity...would you then see anything "sick" about temporarily creating a bunch of copies of yourself and then deleting most of them?

Expand full comment

How would you tell the difference?

Expand full comment

Jesse, you say "work … wouldn’t need to cost much as long as the population of mind uploads included even a small number who were both competent and willing to volunteer their services for free as long as their lives were physically comfortable." If those few were allowed to copy themselves, that would produce subsistence wages for such tasks. Especially <del datetime="2011-12-21T00:19:00+00:00">of sick</del> if such folks "see copies that haven’t diverged too much as fairly disposable." Even in a barter economy - no further "capitalism" is needed. Governments could only limit copies within the scope of their power, and ones that did not would enjoy large competitive advantages. On anthropics, any scenario with a vast future population has similar issues.

Expand full comment

You can't consider a tax rate to be regressive or progressive without specifying what government pays for. Even then, some benefits of government are proportional to wealth (protection from thievery), some are proportional to remaining lifespan (protection from murder).

If ems had the government subsidize electricity prices and also had zero pollution control regulations, biological entities would be adversely affected. There are plenty of people who make arguments for zero pollution regulations now. If the entities voting for and running the government didn't need to breath, eat food, or drink water, how much laxer could the regulations get?

Non-biological entities would consider pollution control regulations to be an unfair and regressive tax on their electricity needs. When ems outnumber biological entities 10,000 to one, the free market would do away with regulations that favor the 0.01% if they add more than 0.01% to the costs of living to the 99.99%.

If 99% of ems are living on subsistence wages, they are not going to tolerate a significant fraction of those subsistence wages going to subsidize a tiny population of biological entities. Especially when much of the time those biological entities are non-productive, during sleep, in utero, before education. Look at the hue and cry over welfare, public education and universal health care now, by entities that are not living on subsistence wages and where a tiny fraction of their taxes go toward those things (and are actually fellow human beings!). What should be expected with ems?

If ems required 3.1 watts of electricity, that is ~25 kwhr/year. “Subsistence” (post tax) would be ~30 kwhr/year. A human requires the land area equivalent of 25,000 kwhr/year to grow food. Would 99% of ems tolerate being taxed at 90% (250/280) to support 1% of humans?

Expand full comment

You say that humans could live off of wealth invested in the em economy. But most humans don't currently have wealth to invest. Are we left to die?

Expand full comment

daedalus2u:

How do you diagnose insanity in an em? Insofar as they are being simulated at a low, neural level:The same way we do with humans: By observing their behavior.

And that sort of filtering will have the same problems we have now.Sometimes even filtering by nature's gold standard, long termfecundity, still gives whole subcultures wearing sacred underwearand believing other batshit insanities.

My point is not that fiddling with neural parameters of ems is safe,but rather that it is fast enough, and likely enough to yieldresults that are more competitive (at either the individual ororganizational levels) that, if not stopped, it will probably move theem population as far from the current human norm as we are fromchimps in a rather short time.

Expand full comment

"Hopefully Anonymous, you are the last person I’d expect to make the objection about lives being worth living. You’ve stated that you’d prefer an immortal life consisting only of torture to a finite life, and expressed bafflement that nearly everyone else doesn’t share that preference."

More basketball-dunking balls in the face. I think my life is worth living, so I should have no objection to a flood of other lives that have a constituent saying they're worth living, in your logical chain. I think this is some sort of positional theater, not a real aesthetic preference of folks like Prof. Hanson. Either way, we're all probably doomed, DOOMED ah-ah-ah-ah-ah.

Expand full comment

All of Robin's original concerns are about long term effects on massive future populations. (Existential risk, etc.) Your concern just involves the humans living (or rather, dying) during the time over which we could delay the em transition, which seems many orders of magnitude less significant.

Expand full comment