80 Comments

and also cheaper. Why would anyone hire you for a day at minimum wage to do something a robot could do for a few pennies worth of electrical power?

To paraphrase the song, “pride and job satisfaction doesn't pay my bills”.

Humans can't survive on pride and job satisfaction, they need food. If humans need to compete for food with ems that can subsist in 1/1000 the amount of power, who is going to get the jobs?

Expand full comment

I think the wallboard installer may be getting more out of it than you realize. I, personally, have derived pride and satisfaction from a long, physically exhausting day spent alphabetizing files, a job which a well-designed robot could probably have done better and faster.

Expand full comment

Several implausible assumptions:

1) Consciousness == free willNot a useful model. Definitions may differ, but imo, these are not synonyms.

2) "unconscious ems" can do all "conscious ems" can do and will be feasible at the same stage of technological progress.If by "unconscious ems" you mean something like narrow AI and expert systems, then they won't have all the abilities ems will have. If not, I don't know what you mean by "unconscious em". General AGI with human-equivalent abilities? Then you break the whole premise of the em scenario, namely that brute-forced human brain emulation comes before such breakthroughs in AI. If that's not true, the whole scenario doesn't materialize in the way Hanson describes.

3) The only preference a rich person can have regarding other conscious entities is sycophantic praiseThis is simply absurd and a function of bad stereotypes on your part, which you're not ready to question. I've already pointed out contradicting examples, and I'm not going to repeat them if you're not willing to read them.

As a final note, of course there's no fundamental reason why conscious beings optimized for sycophantic praise couldn't be happy. Apparently billions are quite happy to kneel before invisible alpha males who don't talk to them. Depending on personality, this is not even a dealbreaker. I'd rather say beware the sadists.

Expand full comment

Yes, if the fact of that outcompetition means the physical destruction of said Nigerians. With EMs, the eventual destruction of the human race is (almost) absolutely certain.

Expand full comment

I do not dispute the claim that some people enjoy some types of work enough to do it all the time and be happy.

Expand full comment

Yes of course, but most jobs can't exist without providing benefits to others - they exist because of gains from trade. Those gains can be the basis for pride and appreciation.

Expand full comment

Jess is right - I think that many real humans today have lives worth living even when their lives are mostly work. This is not because they are deceived about work, but because they actually get satisfaction out of it.

Expand full comment

Are Nigerians who are outcompeted by smart hard-working Chinese justified in doing anything to destroy China?

Expand full comment

as soon as it’s feasible to change the AI software (rather than just simulate a human mind), all sorts of new stuff are possible.You don't even need that much novelty; slight activation threshold shifts in certain (simulated) brain regions like subregions in the nucleus accumbens, pallidum, anterior cingulate, insula and amygdala could do the trick. Have you ever had a day of otherwise boring work when you were in a slightly euphoric state for some reason? Not necessarily a bad experience. If this is combined with benevolent social and legal systems, it might work out just fine.

Expand full comment

Be careful what you wish for. Such ems might not want to stop when the ship was full. Then you would have to program some to unload the ship, which would lead to an endless cycle,which is why work is onerous not pleasurable. Even workaholics complain of their work but are miserable when on vacation,so the go back to work. Creative artists,such as Michelangelo were notoriously miserable,even nearly insane as they worked endlessly ,driven by pain and guilt.Work sucks but idleness is even worse.

Expand full comment

The only reason that people smoke is because they are addicted. If the motivation circuits of ems can be programmed, they can be programmed to be addicted to work of any type. Ems addicted to adding together numbers will add together numbers for subsistence wages.

Ems could be programmed to have ecstatic pleasure from loading cement onto a ship. If their degree of ecstasy was tied to how efficient and low cost they were loading the ship, they might voluntarily turn off their consciousness to save electrical power and be more efficient.

Sacrificing consciousness in the name of efficiency could be like parents sacrificing themselves to save their offspring.

Or like teachers working for substandard wages because they love to teach.

Expand full comment

What I’m interested in is how much we’re allowed to both (a) offload the judgment of the worth-livingness of life onto each individual creature, AND simultaneously (b) put our thumbs on the scale by creating creatures that we know are deceived in the direction of claiming life to be worth living.

I agree this is an important distinction that Robin doesn't highlight enough, so I enjoyed your comment. That said, I think Robin's position is still probably correct.

I suspect that, as Poelmo implies below, states of bliss are neither maintainable long-term in a creature nor compatible with hardworkingness in that creature

Lots to argue with here. First, I don't think we necessarily need to confine ourselves to creatures who are in absolute bliss. I understand your position on the morality of creating typical human, but I think even you should grant that creating lives which are extremely likely to be "pretty good" (an achievable confidence in an em scenario) is morally permissible.

Second, the incompatibility of "pretty good" lives with workaholics is only possibly true if we confine ourselves (as Robin does) to the pure emulation scenario; as soon as it's feasible to change the AI software (rather than just simulate a human mind), all sorts of new stuff are possible.

Even in this highly restricted and short-lived scenario, we can simply get one of these workaholic ems humming for 50 subjective years, burn him out, and then put him in a simulated paradise or whatever he wants after that. (Note that I am not saying here the simulated paradise is the reward for the 50 years of work.) It seems highly likely that the value he produced during those 50 years will easily pay to keep his simulation running into perpetuity.

In what ways is that different from an ordinary atheist worker working all his life toward a fulfillment state of happiness that will never come?

The claim here is that the worker is both enjoying working as it happens and that this enjoyed work has intrinsic value (in the sense that it is morally right for us to create people who so work). Perhaps you're imaging an atheist farmer who labors painfully for decades so that he can have a fulfilling retirement, but this is not what Robin is proposing. I think this is clear from, for example, his post about Jiro Ono the sushi chef.

But if people mainly feel good because they are wrong about something – if they’d be sad if the knew the truth – that seems sad to me, and it seems wrong to create and deceive such creatures

Strongly agree, but I don't think Robin is proposing that. Given the popularly held opinion that certain types of enjoyed hard work have intrinsic value, he's not taking advantage of any ignorance. He's just taking advantage of natural variations in people's temperament.

Expand full comment

Work vs.pleasure.Pleasure: A patient recovering from surgery in the hospital in order to get around no smoking rules gets out of his bed and goes outside of the hospital in the snow to smoke a cigarette.Work: A person puts a hundred bags of cement on a ship.There is some secondary gain. 1.)He is paid.2) He is not beaten. 3.) He believes he is improving his cardiac function. 4.)He gets a prize. A person is more motivated to smoke than work. He can't resist the cigarette but has to force himself to load the cement.

Expand full comment

If it isn't payed for, it isn't "work". If ems are going to be in a work-based economy, what is it that conscious ems can do cheaper than non-conscious ems? I don't think there is anything.

Expand full comment

I don't think you get the point. The cheapest way to do all of those things is not to hire ems that spend computational resources on being conscious.

Unconscious ems will be cheaper at everything than conscious ems will be, except for things that actually require consciousness. The only things that require consciousness, aka free-will, are things like sycophantic praise. The traditional reason that various religious apologists give for the reason that humans have free-will, is so that a free-willed human being could freely choose to worship an omnipotent God who will cause them to burn in infinite Hellfire for an infinite eternity if they don't.

Owners of legacy property will hire ems to generate sycophantic praise or the ems won't be able to pay their electric bill or their memory charges or computational user fees and will be erased to make room for ems that will.

Who will hire conscious ems to do something that non-conscious ems can do better and cheaper? Who is hired these days to add up numbers? No one because non-conscious ems (aka computers) can add numbers faster and cheaper and more accurately than any conscious entity, human or em.

So who is going to hire those trillions of ems and to do what?

Expand full comment

What I'm interested in is how much we're allowed to both (a) offload the judgment of the worth-livingness of life onto each individual creature, AND simultaneously (b) put our thumbs on the scale by creating creatures that we know are deceived in the direction of claiming life to be worth living. Extra-deceived is not the same as extra-happy; I want to keep the distinction clear. I suspect that, as Poelmo implies below, states of bliss are neither maintainable long-term in a creature nor compatible with hardworkingness in that creature. Suffering is a much more powerful motivator; bad is stronger than good.

I bring up the religious example because most of us agree it's sad for a person to suffer all his life for the promise of an imaginary heaven (it's different, as Dremora says, if the person actually experience religious bliss - that might be fine! - but religious bliss, like romantic bliss, tends to wear off). In what ways is that different from an ordinary atheist worker working all his life toward a fulfillment state of happiness that will never come?

I'm not saying these are the only reasons people work. But if people mainly feel good because they are wrong about something - if they'd be sad if the knew the truth - that seems sad to me, and it seems wrong to create and deceive such creatures (though it might be okay to continue to deceive such creatures if they've already been created by others).

Expand full comment