105 Comments

and then erasing the memoriesSorry, I didn't quite address this. If you only counted the memories of experiences, not the experiences themselves, you would run out of luck as soon as it became clear all memories will eventually cease to exist. If the inevitable increase in entropy holds, all of us will definitely one day die, and then all memories of all good and bad experiences will inevitably die with us. Unless you think that the experiences themselves, in those regions of space and time in which they exist, have value, fun shouldn't count at all.

Expand full comment

I was more thinking of the best possible sex or superstimulus narrative, or maybe drug high, but yeah, why not go to an amusement park.

Isn't the reason obvious? You'd do it for the same reason you now do these things: They are fun.

Expand full comment

I consider an em copy "me." However, I consider creating a copy for the purposes of fun and never sharing memories with it equivalent to adding extra years of fun to my life and then erasing the memories of the fun I had during those years.

That might be worthwhile if it's productive fun, like writing a novel or doing science research, but why create an em to read a book or go to an amusement park?

Expand full comment

There is a difference between being "for profit" and being "for rapacious profits".

Expand full comment

I don't believe Sukiyabashi Jiro is a non-profit, thus it is for-profit. But I'm using for-profit in the legalistic/categorical sense, whereas you seem to be talking about personal motivation.

There's a question of the degree to which nominally for-profit organizations actually do have that motivation. Entrepreneurs start businesses because they have a plan, but typically need to promise profits to investors. In a publicly owned corporation there is supposed to be a fiduciary duty to shareholders, but agency problems result in them often being poorly represented. Robin has already mentioned mergers that tend to be bad for the business. Karl Smith used to use Apple as an example of a company that just sat on cash because they refused to pay out dividends, but he can still use Microsoft as an example of a company that wastes money trying to stay relevant and compete with superior companies when they should just expend their assets and cash flow on dividends to their shareholders.

Expand full comment

Out of curiosity, what do you find implausible, the belief that Robin’s scenario is likely to happen, or the his belief that such a scenario is morally good?

The latter of course, the scenario is morally repulsive.

I believe that the “natural regulation” that it is impossible to mass produce people is one of the things that makes libertarian capitalism produce morally positive results. If that regulation is “repealed” by scientific progress the consequences would be disastrous. Hopefully it can be replaced by a legal regulation that restricts em reproduction.

I think a few more regulations than that are needed. Libertarianism is in need of more serious 'philosophical surgery' here ;) In a future of uploads and AIs, a Singleton will have to get control of and regulate the whole environment of natural resources and living things in my view, to prevent not only unchecked population growth, but also catastrophic monoplies of space-time-computational resources and burning of the commons.

The political philosophy of Georgism could be a far more better model for such a future than conventional Libertarianism in my view.

Expand full comment

I think that excellence comes from individuals who want to achieve excellence, not from capitalists who want to make a lot of profit.

Many people are like Jiro, and want to do the best that they can, particularly if it is a job that they love. Much of their satisfaction comes from pride of workmanship, not what they are paid. Capitalists take advantage of this and pay wages below the value of the work because people get personal satisfaction too. This is one of the reasons that teachers are paid such low wages. Some people love to teach children, and will tolerate low wages and terrible conditions and still love to teach children.

What companies are associated with excellence? I can think of only two offhand, Polaroid and Apple.

Both of them were dependent on a CEO who made excellence an important criteria. There are plenty of companies that make more money and who are run by CEOs who get paid a lot more, but excellence isn't measured in terms of profit.

Would Jiro sell sushi that had gone bad? Would a for-profit sushi seller?

Expand full comment

@Evan

I would not want to make copies of myself for the purpose of being happy unless the copies could share the memories of the fun things they did.Well, I would. Apparently you're one of the people who don't include copies into their identity concept - the distinction between those who consider Star Trek style transporting to be a form of death and those who are rational enough to see why it is not would also shape motivations in the em era. Who, of the two mental phenotypes, do you think will spawn the most copies? Who will spawn the happiest? If you're one of the em trillionaires Hanson dreams about, what will you do with the computation? You obviously run yourself as fast as possible, but that has downsides and diminishing returns. You pay other ems to do some stuff, but as mentioned above, there's only so much you can meaningfully have them do without playing absurd zero-sum games. After all, art and design, including that of material objects, can be copied. So an obvious way of investing resources is existence donations to other ems, or creating more copies of yourself who are not doing near subsistence work but enjoying themselves. The majority of ems in the system will be leisure copies of rich people.

Expand full comment

@Dremore

But the type of em that includes copies in their identity concept will be more likely to use resources to create such copies with the purpose of having them be happy.

I would not want to make copies of myself for the purpose of being happy unless the copies could share the memories of the fun things they did. Robin speculates that, while ems will be able to share memories eventually, there will be a long period at the beginning of the Em Era when it is impossible. He also states that his analysis is primarily focused on the pre-memory-sharing era and much of it will not apply once memory sharing is developed.

@mjgeddes

It’s clear the Em scenarios are the product of a Libertarian bias (i.e., the scenarios painted is just Libertarianism carried through to its extreme logical conclusions). It only sounds plausible to ideologes, to most other people its madness (in fact, it’s a good reductio ad absurdum of pure Libertarianism).I am currently a moderate consequentialist libertarian, but I will become much less libertarian when ems are invented because I agree with you that Robin's scenario is mad. I believe that the "natural regulation" that it is impossible to mass produce people is one of the things that makes libertarian capitalism produce morally positive results. If that regulation is "repealed" by scientific progress the consequences would be disastrous. Hopefully it can be replaced by a legal regulation that restricts em reproduction.

Out of curiosity, what do you find implausible, the belief that Robin's scenario is likely to happen, or the his belief that such a scenario is morally good? I find Robin's scenario to be morally awful, but believe that it is very technologically plausible and a likely path for the future development. The only assumption of his I find at all implausible is his belief that there will be a relatively long period of time between the invention of ems and the invention of things like memory-merging, sideloading, and de novo AI.

Expand full comment

You're right, he does mention giving away for free, which is different from the em scenario he mentions later where they are working for subsistence wages. I'm now puzzled why he introduced the zero-price hypothetical.

It's funny that you bring up I.E, because my windows computers have continued to come with it pre-installed, yet its user share has dropped like a rock!

Where do you think excellence tends to come from?

Expand full comment

Jiro giving away his sushi for free was part of Hansen's scenario.

First, imagine that Jiro is not rich. He is still the very best, but he gives his sushi away. He has enough to eat, stays warm, and is healthy, but has few luxuries. But since he spends most of his time at the office, it probably doesn’t make that much difference to his quality of life if he is rich or poor.

There are issues with giving away things for free. Google recently lost an unfair trade lawsuit in France brought by map sellers because it gave google-maps away for free, and was ordered to pay damages. M$ giving Internet Explorer away for free was part of their business strategy to maintain their monopoly in operating systems and extend it to the browser market. By giving IE away for free and subsidizing it with profits from their OS business, they would prevent any for profit entity from entering the browser market.

For profit producers usually don't strive for excellence, they strive for monopoly power. You can get monopoly power through excellence, but it is a lot easier to get it through collusion, trickery, fraud, bribery, co-option of regulators, acquisition of competitors or other unfair trading practices.

Expand full comment

He is not trying to secretly slip pro-rich people propaganda into this post. Or any of his posts for that matter.

I personally think Hanson is yet another example of a very smart person who believes very weird things. It's clear the Em scenarios are the product of a Libertarian bias (i.e., the scenarios painted is just Libertarianism carried through to its extreme logical conclusions). It only sounds plausible to ideologes, to most other people its madness (in fact, it's a good reductio ad absurdum of pure Libertarianism).

Expand full comment

I doubt ems that see deletion as no big deal will have an easy time finding jobs.But the type of em that includes copies in their identity concept will be more likely to use resources to create such copies with the purpose of having them be happy. If total wealth increases (as Hanson suggests) and ems work for others in a market with property rights, then there must be stuff-owners (Hanson imagines trillionaires) who are swimming in wealth. Those are most likely to use that wealth to spawn leisure copies of themselves. Imagine you live in that era and you are very wealthy. What else would you do with the resources?

Expand full comment

Robin says:

People now have a wide range of attitudes to a Star Trek style transporter – some think it a horrible death, and others see it as like walking through a door. People can have a similarly wide range of attitudes toward temporary copies. It is those who think it no big deal that will populate the em world.I doubt that very much. Who do you think will work harder, a person who thinks it's no big deal if they are deleted because another copy exists somewhere, or a person who is terrified of being deleted because they think it will kill them? I know that if I was a ruthless and heartless corporation I'd hire ems who thought deletion was a horrible death. They'd be so terrified of being deleted that they'd work themselves to the bone.

I doubt ems that see deletion as no big deal will have an easy time finding jobs. There'd just be too much competition from the haggard, terrified ems that are willing to work themselves to the point of death to avoid being erased.

@Sister Y:

It only seems necessary to feel superior to others, not to actually be superior by some objective measure. So the cheery optimist might propose that everyone could subjectively feel superior to someone by lying to themselves – and self-deception in that direction does seem to be required for happy mental functioning. It seems at counter purposes to the mission of this blog, though.It isn't necessary for people to deceive themselves. All they need to do is feel superior to other people at something that they deem important while the other people are superior at something else they deem important. They don't need to feel superior to others in every conceivable way. All we need to do is make sure everyone feels superior is for some people to be superior at X and inferior in Y and other people to be superior at Y and inferior in X.

The Robber's cave experiment provides a good example of this. One group of boys thought they were superior at being tough and manly, the other group thought they were superior at being pure and religious. Both were objectively superior in those attributes, and both derived meaning from their superiority. The fact that neither group had absolute superiority over the other did not bother them.

@TGGP

Evan, the Repugnant Conclusion comes from repeated steps. At every step you have an incremental decision to make. So what steps would you take and when?I would determine the point where population will reach such a size that drain each person places on their society's wealth will soon begin to exceed the amount of new wealth they create by increasing division of labor. I would establish that as a Schelling Point and stop population growth there.

It's no different from stopping littering. Each person has an incremental decision to make, and each of those decisions won't do a lot of harm by itself (honestly, what's just one more bottle on the side of the road?). But those millions of incremental decisions added up cause a bad scenario no one wants (roadsides covered in garbage). So society establishes a Schelling point (don't litter at all, in this case).

In the Mere Addition Paradox itself, I would simply deny that A+ is necessarily better than A, since the two populations added together would have both lower average utility and lower equality of utility than A.

In terms of resource percentages, I haven't got an exact number nailed down, but as a ballpark estimate based on intuition, I would say that if a society gains control of some new resources, it should use no more than 50% of those resources to create new people and no more than 90% of them to improve the lives of existing people. In other words, spend 90%-50% on average utility, 10%-50% on total utility. It may be acceptable to spend more on increasing the population if the population is so small it is at risk of extinction but otherwise increasing quality of life is at least as important as increasing quantity.

@daedelus2u

My explanation is that this discussion isn’t really about entities like Jiro, it is about what is the minimum level of quality of life that the 99% can be allowed to have, as if the only threshold that matters is whether the life is worth living, or not.No, Robin is making speculations about the moral nature of a hypothetical future society. He is not trying to secretly slip pro-rich people propaganda into this post. Or any of his posts for that matter. I think you are just really interested in talking about the negative behavior of rich people and are reading this into Robin's post because it gives you an opportunity to expound on it.

Reasoning that “even overworked slaves want to remain alive”, does not justify working entities so hard that they are on the verge of being suicidal.I agree with you here. But Robin does not. He believes that as long as most people have enough resources that their lives are "barely worth living" that it is always morally better to use excess resources to create new people, rather than improve the lives of existing people.

Robin holds this belief for philosophical reasons, not because he is a shill for the rich. If I were to attribute any ulterior motive to him it would be that his beliefs allow him to make many novel, unique, and unorthodox moral arguments that signal much more intelligence than a more normal moral position would. But even in that case, his views should be evaluated independently of the history of their formation.

Expand full comment

True happiness is finding your highest value, which is generally where you are most appreciated, because this necessarily maximizes your potential. Now, one is probably never on their 'global maximum', but local max is pretty good. I suppose any skill that can be described really well can in theory be replaced by a machine, but a lot of life is timing: specific skills have very different values each decade, but general skills are more stable.

I do think monomaniacal experts are not as interesting or worthy of emulation as those who can see the context around them, appreciating others, etc. Moderation in all things.

The universe is infinite and individuals are finite, so if you want to matter to the universe as an individual you are doomed. You don't have to be essential to be important, but you do have to matter to some people. Striving to be above average is therefore a good intermediate goal.

Expand full comment

Poelmo, Robin Hanson has a paper on burning the cosmic commons. There is no 0.1% aristocracy, just masses of hardscrapple Malthusian ems. Any entity that doesn't expend all its "wealth" in maximizing reproduction is outbred by those who do.

Evan, the Repugnant Conclusion comes from repeated steps. At every step you have an incremental decision to make. So what steps would you take and when?

daedalus2u, I am confused as to why you have Jiro giving away sushi for free, since that's neither done by the actual Jiro nor Hanson's hypotheticals. In the absence of price rationing you are right to expect line-waiters. But we don't see your scenario too often in real life because for-profit producers endeavor to get good reviews and spread word-of-mouth. They might give out free samples, and in that case people would find out that the line-waiters were lying. Also how often do people just cease eating voluntarily and die? Anorexics might die inadvertently, but others I'd expect to commit suicide in a quicker way.

Poelmo, good point about trying to fill low-status jobs. That's why I agree with Jeffrey Friedman's "There Is No Substitute For Profit and Loss".Regarding ems killing humans, Robin has previous said we can rely on the legal system. But it would be rather tempting for ems and it's likely they would come to develop an ingroup identity which excluded meatbags. On the other hand, they might become "bred" for domestication.

daedalus2u, the reason the focus is on the creation of more entities is because that's what Hanson expects natural selection to produce in the absence of a singleton, because he doesn't think our coordination ability is high enough to produce a singleton.

Expand full comment