Parents sometimes disown their children, on the grounds that those children have betrayed key parental values. And if parents have the sort of values that kids could deeply betray, then it does make sense for parents to watch out for such betrayal, ready to go to extremes like disowning in response.
"More clock cycles would make an EM faster, maybe cleverer, but not wiser."
If you can buy more clock cycles you can also buy more memory. Combine these things and you have a mind that keep tracks of many more details, processes them faster and recognizes relations between things more quickly. If that does not increase wisdom then nothing does. Plus, such an EM will do ordinary tasks much quicker, that alone is enough to start a snowball of accumulation of currency followed by the purchase of more clock cycles and memory followed by even faster accumulation of currency followed by the purchase of even more clock cycles and memory and so on... Just look how comparable small differences in effort and natural ability, aided by luck, already produce the difference between wage slaves and billionaires today, that's nothing compared to what would happen if you could vastly improve your mind with money. Yes, I know that's hard to imagine but it was also hard to imagine for a cave man that someday people would gas millions of women and children in the span of a few years and that the richest men would own many thousands of times more than the average man, yet these things happened. Robin asks us to not close our mind for the possibilities of the future, I think it's only fair if that includes the possible horrors of the future if we, for some stupid reason, put aside all our ethics and human rights, but choose to stick with an obsolete economical system.
More clock cycles would make an EM faster, maybe cleverer, but not wiser. A fool who can put the rest of the world on slo-mo (for a billion dollars per second) is still a fool. Living a thousand years, murdering everyone who points and says "You're a fool!" just makes 'em a fool with enemies who are smart enough to plan quietly.
In more technical terms, mind emulation doesn't change the way comparative advantage works.
In a world of EMs, when you need a plumber, you look up the Greatest Plumber Who Has Ever Lived, who takes a fraction of a second (from the mortal perspective) to glance at the situation before referring you to the God-King of the specific sort of plumbing problem you have. Every sensible child dreams of growing up to be the digitized god of some yet-undiscovered professional specialty.
"One Bushman, when asked why he hadn’t emulated neighboring tribes by adopting agriculture, replied, “Why should we, when there are so many mongongo nuts in the world?” "Rather than this being some backwards conservatism the bushman actually has a point. He is likely to live longer, healthier, happier and more free as a bushman than as an African peasant or slum dweller. Only if he moved to the West he could improve his quality of life, but what are the chances of that happening? So it was for ancient hunter-gatherers as well: becoming farmers did not make their lives better and they could not have anticipated modern technology, democracy and medicine so they were right to prefer their own way of life. The only disadvantage was that farming societies could raise bigger armies and that's why there are so few hunter gatherer's left (10 malnourished conscripts without teeth will still beat 1 healthy warrior).
"One Bushman, when asked why he hadn’t emulated neighboring tribes by adopting agriculture, replied, “Why should we, when there are so many mongongo nuts in the world?”"
Yes, and how many bushmen are left?
And while some farmers certainly choose freely to move to cities, others do so because they have no choice
Nevertheless, it is not considered a dishonourable behavior.
In general I agree with you. That is where most disowning I've heard about seems to come from. But what if your child became an unrepentant mass murderer and serial rapist, or a brutal dictator? Would disowning be a rational response then?
Anecdotal evidence from Jared Diamond: "One Bushman, when asked why he hadn’t emulated neighboring tribes by adopting agriculture, replied, “Why should we, when there are so many mongongo nuts in the world?”"
And while some farmers certainly choose freely to move to cities, others do so because they have no choice – their compatriots who chose to adopt industrial methods have made their farming unnecessary, and eliminated or undercut their market. I'd be curious to see any research on the strength of the different driving forces here.
"If the probability of finding anything else becomes exceedingly low then you have no choice, just as in official slavery."
Sure you do: You can try to find another job, find charity, or choose painless death.
"No entity will suffer if it has never existed."
No entity has to suffer if it can choose painless death instead.
"Even if you think thetwo are equal then you cannot live by it (because you being alive meansyou already took resources that could've gone to an unborn entity),it's really unworkable."
What do you mean, live by it? You don't have to give your resources to others, you don't have to have children and you don't have to copy yourself if you're an em.
What I object to is that others should use violence to prevent you from using your resources in these ways, maybe with the exception of having children who will suffer and can't make choices for themselves for at least a few years (this is why I argue that the poor don't have a "basic right" to reproduce freely, a position which of course is usually met with rejection from most people).
"The fact is that if I don't believe my copy is me than I can abuse himand he wouldn't be comforted by some philosophical mumbo jumbo that heis actually me"
Sure and the same can be said for ruining your health and burning your money, leaving your future self to rot. You're free to do so if you like, it's a free world.
"A 1000x EM won't trade with a 1x EM, he will buy his land for glassbeads (metaphorically speaking) and at the same time make the 1x EMthink the deal was his own idea, then the 1000x EM will buy back hisglass beads for plastic beads."
I think this is nonsense, but even if we assume that inequality leads to information asymmetries that can skew decisions in an exploitative way, surely it still beats top-down coercion?
As they say, the guy who tries to ban your job (or life!) isn't your friend.
"The vast majority of EMs would not have leisure time, be able to choosea profession or a hobby or speak their mind freely and there would beno privacy at all."
I really hate it when people make stuff up and add it to a general scenario discussion.
What is the primary destructive factor for free speech and privacy today? Is it wealth inequality?
No.
It's top-down government coercion, of course.
You're also ignoring that total wealth would increase, and this means much more total leisure time.
"1) I would not equate voluntary subsistence work with slavery; imo the crucial difference is the ability to reject it without suffering (even if that means death if you can't find anything else)"If the probability of finding anything else becomes exceedingly low then you have no choice, just as in official slavery.
"2) I see no good reason why never coming into existence is better than painless death after coming into existence."
No entity will suffer if it has never existed. Even if you think the two are equal then you cannot live by it (because you being alive means you already took resources that could've gone to an unborn entity), it's really unworkable.
"3) The consent of a person who copies is a stronger from of consent than that of a parent who has a child the natural way; copying a fully mature mind is also much less intrusive than focing a child to live dependent for years - if you reject one, you must reject the other even more."
I do not reject creating EMs, I reject creating EMs carelessly, knowing they won't have good lives, like I said, if the creator was made responsible to get the EM on its feet or if there was a welfare program for EMs all would be fine.
"4) Personal identity is not as binary as you imply it is; if you think the person who wakes up tomorrow is you (or at least somewhat-you), you may also think that a copy of your brain would be you (or at least somewhat-you)."
The fact is that if I don't believe my copy is me than I can abuse him and he wouldn't be comforted by some philosophical mumbo jumbo that he is actually me and therefore somehow experiences my life of unbridled debauchery too, even though he really can't remember setting foot on my pleasure island after he awoke from the copying process.
"5) Non-zero-sum games do not require benevolence, they can rely on mutual benefit and transparency alone. Trade for mutual benefit doeswork even if one party is much richer."
Again, you are projecting current flesh and blood matters onto an EM future that's radically different. A 1000x EM won't trade with a 1x EM, he will buy his land for glass beads (metaphorically speaking) and at the same time make the 1x EM think the deal was his own idea, then the 1000x EM will buy back his glass beads for plastic beads. 1x EMs will be nothing more than playthings for the 1000x EMs, there just will not be anything even remotely resembling a level playing field. The 1000x EM will never die so there will be no stupid heirs spoiling his money either. Once an EM gets a runspeed advantage over the others he is set for eternity because the others won't be able to compete, they simple lack the processing power (essentially your condition of "transparency" is violated). Social mobility will not exis in such a society.
"8) Communism forced people to starve and replaced incentives with increasing threats of violence, which is the alternative currency to allowing consent-based trade and liberty. Judge for yourself how much of that you want."
Few people starved in the Soviet Union after Stalin died, certainly not more then were starving in the Mississippi delta at the same time. There was not much liberty in the Soviet Union but there would be even less in Hanson's EM society. The vast majority of EMs would not have leisure time, be able to choose a profession or a hobby or speak their mind freely and there would be no privacy at all. Once you look beyond the theory it should be obvious how a little of real liberty that you can exercise is better than a lot of theoretical freedom that you cannot exercise. In fact the 1000x EMs wouldn't even need you to do jobs for them, there's nothing you have that they would want, they can simply prey on the desperate until they have so much power that they could start systematic purges, destroying businesses and jobs until all all non-1000x EMs had "starved" to death and they control all the resources.
"even though you describe non-violent people chilling in luxurious paradises, you seem to imply this is a bad thing rather than a good thing."
It's a good thing if everyone had a reasonable chance at obtaining such a life, for example if it was based on reversible meritocracy, while at the same time maintaining some decent minimum standard for those who lost the race. But that's not the case here.
"9) I have no idea - none - why you think Hanson's futurism is inconsistent unless he has 20 kids or commits suicide (what is this I don't even)"
Because what's good for the goose is good for the gander. By now Hanson has made it blatantly obvious that he desires the EM society he portrays. He should lead by example (increase the number of humans on the planet, or make room for them) because the world doesn't need another NIMBY/chickenhawk.
1) I would not equate voluntary subsistence work with slavery; imo the crucial difference is the ability to reject it without suffering (even if that means death if you can't find anything else)
2) I see no good reason why never coming into existence is better than painless death after coming into existence.
3) The consent of a person who copies is a stronger from of consent than that of a parent who has a child the natural way; copying a fully mature mind is also much less intrusive than focing a child to live dependent for years - if you reject one, you must reject the other even more.
4) Personal identity is not as binary as you imply it is; if you think the person who wakes up tomorrow is you (or at least somewhat-you), you may also think that a copy of your brain would be you (or at least somewhat-you).
5) Non-zero-sum games do not require benevolence, they can rely on mutual benefit and transparency alone. Trade for mutual benefit does work even if one party is much richer.
7) Your position correctly points out negative externalities but ignores all positive externalities such as capital value increases; even though you describe non-violent people chilling in luxurious paradises, you seem to imply this is a bad thing rather than a good thing.
8) Communism forced people to starve and replaced incentives with increasing threats of violence, which is the alternative currency to allowing consent-based trade and liberty. Judge for yourself how much of that you want.
9) I have no idea - none - why you think Hanson's futurism is inconsistent unless he has 20 kids or commits suicide (what is this I don't even)
P.S. If having more beings alive is the greatest good then why leave it up to the decisions of existing persons, why not have an automatic EM creation programme creating new EMs every time the total processing power is increased and allotting a subsistence level of hardware resources to each EM (having one 1000x EM prevents 999 other EMs from existing)? I want to keep hammering the point that Hanson is willing to throw many human rights under the bus of utility but the "right to collect way more crap than you need to be happy" (and only for a privileged few, who are often not even the most productive members of society, at that) is somehow preserved. Shouldn't he be old enough to know that you can't buy love, happiness or dignity?
"The original consented to both; this is like saying when I go to sleep today I should be painlessly killed, because my tomorrow's self did not consent to waking up."No, the original could be chilling on a luxurious island paradise and choose for the copy to become an enslaved drone even though the original would never want to become an enslaved drone himself. The copy and the original are two different people, they share the same memories but they are not the same persons from the moment of the copying onwards.
"You're missing the point, which is that children never consent to being born, which was your entire objection in this specific part of the argument."
My point was that although children do not choose to be born society forces the parents to either take good care of their children or hand them over to someone who will. This is perfectly possible to implement in an EM society as well, but Hanson chose not to include it in his version. Even if he would you waould run into problems because EM population explosions could happen in miliseconds whereas in the real world people get the chance to slowly experience the growing effects of overpopulation and change their behavior accordingly, taking multiple generations if need be (while EM society could get stuck in destructive traditions because there is no natural death to make room for open minded new generations).
"Painless death is fine - after all, you want to force it onto them by banning the technology use that would give them the option to live in the first place."
You are equating getting killed with never having lived. It's just not the same thing, and if it was you'd be damned if you do and damned if you don't, so you might as well pick the choice that gives people the highest quality of living. Besides, I have yet to see anything acting on the idea: Robin Hanson still does not have 20 kids, still has not given up his worldly possessions or committed suicide so the Earth can support more people.
"No, it really isn't."
So you really think life in the Soviet Union in the 1960s was worse than being some drone in a machine forced to slave away at subsistence level for an eternity, and getting killed if you slip up once?
"You assume that inequality is the same thing as slavery, but it is not; slavery takes your rights away while inequality simply means you have less resources than others."
It doesn't matter in practice if quitting my job results in me being put before the firing squad or starving to death in the gutter. If I cannot effectively control the course of my life, no matter how hard I work, then I am a slave.
"Not all games are zero-sum!"
Are you implying the 1000x EMs would suddenly grow a conscience? They can grow the pie a thousand times, but they're not at all obliged to share any of that growth with the rest. There is no competition, no free market when some people can buy radically faster brains and eternal life, capitalism doesn't simply doesn't work anymore, perhaps communism won't either, but there already exist alternatives to both and undoubtedly more will be invented by future thinkers.
"The original consented to being copied, not to become a copy"
The original consented to both; this is like saying when I go to sleep today I should be painlessly killed, because my tomorrow's self did not consent to waking up.
This is your reasoning to prevent life of intelligent beings who want to live?
"Normal reproduction os limited (a woman can only have so many children)"
You're missing the point, which is that children never consent to being born, which was your entire objection in this specific part of the argument.
"...then they will likely face enslavement or death."
Painless death is fine - after all, you want to force it onto them by banning the technology use that would give them the option to live in the first place.
"
Radical communism is heaven compared to Hanson's EM future, unless you are one of those few 1000x EMs..."
No, it really isn't.
The intuition you are holding that power abuse can induce suffering is exactly the reason why I reject top-down authoritarian morality.
You assume that inequality is the same thing as slavery, but it is not; slavery takes your rights away while inequality simply means you have less resources than others.
"1) We know pretty well what the em would want, because it is a copy of the individual who consents just moments prior."The original consented to being copied, not to become a copy who has to start life all over again for him/herself with nothing but the ones and zeros on their back.
"2) This radical requirement of absolute autonomy has never and will never be implemented; normal reproduction is much more imposing on the offspring since they require years of development before becoming intellectually mature."
Normal reproduction os limited (a woman can only have so many children) and the government steps in when children are abused, at least in civilized countries. In Hanson's world there will be no welfare programs to help newly created EMs get on their feet, they will just be thrown into the deep, if they are unlucky enough that no one left them anything and no one wants to take care of them then they will likely face enslavement or death.
"You assume ems will be sold as property; I assume they will be self-owned persons."
Hanson talks about EMs created for specific tasks and even if there's no official slavery there will be unofficial slavery serving the 1000x CPU power EM elite.
"3) Monocultures are vulnerable in specific ways; they would have to diversify or cooperate with others or their power base crashes."
Why? There's no physical violence and they are always literally 999 steps ahead of everyone else, and they can keep buying more CPU power, a 1x EM will never be able to catch up to them.
"4) I concede the point about externalities and I have no good solution except to say that freedom doesn't have to be all-or-nothing, and top-down bans on individual freedom have historically never led to good outcomes (compare radical communism)."
Radical communism is heaven compared to Hanson's EM future, unless you are one of those few 1000x EMs (but it's a strawman to immediately compare any rules that protect the weak to radical communism). The level of suffering in Hanson's world is almost incomprehensible to us, radical communism may have looked like a big deal in the past, but it's nothing compared to what the future can bring, so we should be careful about trying to run away from it so fast that we invite something even worse. The danger is that we will underestimate the possibilities of the future, both negative (potential for suffering if we apply obsolete concepts to a future world) and positive (the many ways smart thinking and advanced technology could make the average life more pleasant if we allow our economic thinking to evolve with our technology, relatively tiny infractions on the personal freedoms of the elite who are making much more than what their efforts justify, like setting a maximum wage of $1 million, could free countless souls from need and want).
"The EM is a separate entity, no one has the right to decide for it."
1) We know pretty well what the em would want, because it is a copy of the individual who consents just moments prior.
2) This radical requirement of absolute autonomy has never and will never be implemented; normal reproduction is much more imposing on the offspring since they require years of development before becoming intellectually mature.
"Youshouldn't sell an EM, just like you shouldn't sell a child."
You assume ems will be sold as property; I assume they will be self-owned persons.
I also assume they can shut themselves down painlessly whenever they want (there could be laws about that)
If you assume slavery, the whole debate becomes about slavery, not economics + futurism.
Just fight slavery.
"You are also forgetting that after a few EMs have bought 1000x CPU power they are essentially unstoppable"
1) Monocultures are vulnerable in specific ways; they would have to diversify or cooperate with others or their power base crashes.
2) I concede the point about externalities and I have no good solution except to say that freedom doesn't have to be all-or-nothing, and top-down bans on individual freedom have historically never led to good outcomes (compare radical communism).
Yeah (got your reference about a certain person's interest in cryogenics), it's NIMBYism to the extreme. These people never put their money (or their lives) where their mouth is, they're like chickenhawks, but even worse, preaching massive suffering, as long as it's someone else suffering.
"A person consents to scan his brain, becoming the initial em."The EM is a separate entity, no one has the right to decide for it. You shouldn't sell an EM, just like you shouldn't sell a child.
"There is never a point where authoritarian-utilitarian Hanson decrees that they should live in misery because it "always beats never having existed"."
Yes there is such a point: the EMs are created by someone who can afford not to share in their misery, yet Hanson (and you) argues that there should be no limits placed on copying. Essentially that's the same as saying children are their parent's property and parents don't have to look after their own children, that having laws against child abuse is wrong. It's also BS from a utalitarian viewpoint because if everything is subservient to utility, including the quality of life of the copies, then why go out of your way for some mundane "freedom to copy"?
You are also forgetting that after a few EMs have bought 1000x CPU power they are essentially unstoppable (they are not even mortal), any EMs created after that point will never be able to compete with the 1000x EMs (they think 1000x faster than a normal EM), who will then use their power to buy another 1000x CPU power and so on until the same handful of EMs dominate all the others forever, they will be the only ones for which the market will be free, everyone else will be a slave in all but name. Holding on to capitalism in such circumstances epitomises obsolete thinking which Hanson says he's against.
All in all it's quite a dystopian future Hanson is showing us, and I don't think it'll ever come to pass without some serious, violent resistance because many humans, and AIs, would definitely prefer death over Hanson's future.
"More clock cycles would make an EM faster, maybe cleverer, but not wiser."
If you can buy more clock cycles you can also buy more memory. Combine these things and you have a mind that keep tracks of many more details, processes them faster and recognizes relations between things more quickly. If that does not increase wisdom then nothing does. Plus, such an EM will do ordinary tasks much quicker, that alone is enough to start a snowball of accumulation of currency followed by the purchase of more clock cycles and memory followed by even faster accumulation of currency followed by the purchase of even more clock cycles and memory and so on... Just look how comparable small differences in effort and natural ability, aided by luck, already produce the difference between wage slaves and billionaires today, that's nothing compared to what would happen if you could vastly improve your mind with money. Yes, I know that's hard to imagine but it was also hard to imagine for a cave man that someday people would gas millions of women and children in the span of a few years and that the richest men would own many thousands of times more than the average man, yet these things happened. Robin asks us to not close our mind for the possibilities of the future, I think it's only fair if that includes the possible horrors of the future if we, for some stupid reason, put aside all our ethics and human rights, but choose to stick with an obsolete economical system.
More clock cycles would make an EM faster, maybe cleverer, but not wiser. A fool who can put the rest of the world on slo-mo (for a billion dollars per second) is still a fool. Living a thousand years, murdering everyone who points and says "You're a fool!" just makes 'em a fool with enemies who are smart enough to plan quietly.
In more technical terms, mind emulation doesn't change the way comparative advantage works.
In a world of EMs, when you need a plumber, you look up the Greatest Plumber Who Has Ever Lived, who takes a fraction of a second (from the mortal perspective) to glance at the situation before referring you to the God-King of the specific sort of plumbing problem you have. Every sensible child dreams of growing up to be the digitized god of some yet-undiscovered professional specialty.
"One Bushman, when asked why he hadn’t emulated neighboring tribes by adopting agriculture, replied, “Why should we, when there are so many mongongo nuts in the world?” "Rather than this being some backwards conservatism the bushman actually has a point. He is likely to live longer, healthier, happier and more free as a bushman than as an African peasant or slum dweller. Only if he moved to the West he could improve his quality of life, but what are the chances of that happening? So it was for ancient hunter-gatherers as well: becoming farmers did not make their lives better and they could not have anticipated modern technology, democracy and medicine so they were right to prefer their own way of life. The only disadvantage was that farming societies could raise bigger armies and that's why there are so few hunter gatherer's left (10 malnourished conscripts without teeth will still beat 1 healthy warrior).
"One Bushman, when asked why he hadn’t emulated neighboring tribes by adopting agriculture, replied, “Why should we, when there are so many mongongo nuts in the world?”"
Yes, and how many bushmen are left?
And while some farmers certainly choose freely to move to cities, others do so because they have no choice
Nevertheless, it is not considered a dishonourable behavior.
In general I agree with you. That is where most disowning I've heard about seems to come from. But what if your child became an unrepentant mass murderer and serial rapist, or a brutal dictator? Would disowning be a rational response then?
Anecdotal evidence from Jared Diamond: "One Bushman, when asked why he hadn’t emulated neighboring tribes by adopting agriculture, replied, “Why should we, when there are so many mongongo nuts in the world?”"
And while some farmers certainly choose freely to move to cities, others do so because they have no choice – their compatriots who chose to adopt industrial methods have made their farming unnecessary, and eliminated or undercut their market. I'd be curious to see any research on the strength of the different driving forces here.
"If the probability of finding anything else becomes exceedingly low then you have no choice, just as in official slavery."
Sure you do: You can try to find another job, find charity, or choose painless death.
"No entity will suffer if it has never existed."
No entity has to suffer if it can choose painless death instead.
"Even if you think thetwo are equal then you cannot live by it (because you being alive meansyou already took resources that could've gone to an unborn entity),it's really unworkable."
What do you mean, live by it? You don't have to give your resources to others, you don't have to have children and you don't have to copy yourself if you're an em.
What I object to is that others should use violence to prevent you from using your resources in these ways, maybe with the exception of having children who will suffer and can't make choices for themselves for at least a few years (this is why I argue that the poor don't have a "basic right" to reproduce freely, a position which of course is usually met with rejection from most people).
"The fact is that if I don't believe my copy is me than I can abuse himand he wouldn't be comforted by some philosophical mumbo jumbo that heis actually me"
Sure and the same can be said for ruining your health and burning your money, leaving your future self to rot. You're free to do so if you like, it's a free world.
"A 1000x EM won't trade with a 1x EM, he will buy his land for glassbeads (metaphorically speaking) and at the same time make the 1x EMthink the deal was his own idea, then the 1000x EM will buy back hisglass beads for plastic beads."
I think this is nonsense, but even if we assume that inequality leads to information asymmetries that can skew decisions in an exploitative way, surely it still beats top-down coercion?
As they say, the guy who tries to ban your job (or life!) isn't your friend.
"The vast majority of EMs would not have leisure time, be able to choosea profession or a hobby or speak their mind freely and there would beno privacy at all."
I really hate it when people make stuff up and add it to a general scenario discussion.
What is the primary destructive factor for free speech and privacy today? Is it wealth inequality?
No.
It's top-down government coercion, of course.
You're also ignoring that total wealth would increase, and this means much more total leisure time.
"1) I would not equate voluntary subsistence work with slavery; imo the crucial difference is the ability to reject it without suffering (even if that means death if you can't find anything else)"If the probability of finding anything else becomes exceedingly low then you have no choice, just as in official slavery.
"2) I see no good reason why never coming into existence is better than painless death after coming into existence."
No entity will suffer if it has never existed. Even if you think the two are equal then you cannot live by it (because you being alive means you already took resources that could've gone to an unborn entity), it's really unworkable.
"3) The consent of a person who copies is a stronger from of consent than that of a parent who has a child the natural way; copying a fully mature mind is also much less intrusive than focing a child to live dependent for years - if you reject one, you must reject the other even more."
I do not reject creating EMs, I reject creating EMs carelessly, knowing they won't have good lives, like I said, if the creator was made responsible to get the EM on its feet or if there was a welfare program for EMs all would be fine.
"4) Personal identity is not as binary as you imply it is; if you think the person who wakes up tomorrow is you (or at least somewhat-you), you may also think that a copy of your brain would be you (or at least somewhat-you)."
The fact is that if I don't believe my copy is me than I can abuse him and he wouldn't be comforted by some philosophical mumbo jumbo that he is actually me and therefore somehow experiences my life of unbridled debauchery too, even though he really can't remember setting foot on my pleasure island after he awoke from the copying process.
"5) Non-zero-sum games do not require benevolence, they can rely on mutual benefit and transparency alone. Trade for mutual benefit doeswork even if one party is much richer."
Again, you are projecting current flesh and blood matters onto an EM future that's radically different. A 1000x EM won't trade with a 1x EM, he will buy his land for glass beads (metaphorically speaking) and at the same time make the 1x EM think the deal was his own idea, then the 1000x EM will buy back his glass beads for plastic beads. 1x EMs will be nothing more than playthings for the 1000x EMs, there just will not be anything even remotely resembling a level playing field. The 1000x EM will never die so there will be no stupid heirs spoiling his money either. Once an EM gets a runspeed advantage over the others he is set for eternity because the others won't be able to compete, they simple lack the processing power (essentially your condition of "transparency" is violated). Social mobility will not exis in such a society.
"8) Communism forced people to starve and replaced incentives with increasing threats of violence, which is the alternative currency to allowing consent-based trade and liberty. Judge for yourself how much of that you want."
Few people starved in the Soviet Union after Stalin died, certainly not more then were starving in the Mississippi delta at the same time. There was not much liberty in the Soviet Union but there would be even less in Hanson's EM society. The vast majority of EMs would not have leisure time, be able to choose a profession or a hobby or speak their mind freely and there would be no privacy at all. Once you look beyond the theory it should be obvious how a little of real liberty that you can exercise is better than a lot of theoretical freedom that you cannot exercise. In fact the 1000x EMs wouldn't even need you to do jobs for them, there's nothing you have that they would want, they can simply prey on the desperate until they have so much power that they could start systematic purges, destroying businesses and jobs until all all non-1000x EMs had "starved" to death and they control all the resources.
"even though you describe non-violent people chilling in luxurious paradises, you seem to imply this is a bad thing rather than a good thing."
It's a good thing if everyone had a reasonable chance at obtaining such a life, for example if it was based on reversible meritocracy, while at the same time maintaining some decent minimum standard for those who lost the race. But that's not the case here.
"9) I have no idea - none - why you think Hanson's futurism is inconsistent unless he has 20 kids or commits suicide (what is this I don't even)"
Because what's good for the goose is good for the gander. By now Hanson has made it blatantly obvious that he desires the EM society he portrays. He should lead by example (increase the number of humans on the planet, or make room for them) because the world doesn't need another NIMBY/chickenhawk.
IMASBA
Final addressing some of your points:
1) I would not equate voluntary subsistence work with slavery; imo the crucial difference is the ability to reject it without suffering (even if that means death if you can't find anything else)
2) I see no good reason why never coming into existence is better than painless death after coming into existence.
3) The consent of a person who copies is a stronger from of consent than that of a parent who has a child the natural way; copying a fully mature mind is also much less intrusive than focing a child to live dependent for years - if you reject one, you must reject the other even more.
4) Personal identity is not as binary as you imply it is; if you think the person who wakes up tomorrow is you (or at least somewhat-you), you may also think that a copy of your brain would be you (or at least somewhat-you).
5) Non-zero-sum games do not require benevolence, they can rely on mutual benefit and transparency alone. Trade for mutual benefit does work even if one party is much richer.
7) Your position correctly points out negative externalities but ignores all positive externalities such as capital value increases; even though you describe non-violent people chilling in luxurious paradises, you seem to imply this is a bad thing rather than a good thing.
8) Communism forced people to starve and replaced incentives with increasing threats of violence, which is the alternative currency to allowing consent-based trade and liberty. Judge for yourself how much of that you want.
9) I have no idea - none - why you think Hanson's futurism is inconsistent unless he has 20 kids or commits suicide (what is this I don't even)
P.S. If having more beings alive is the greatest good then why leave it up to the decisions of existing persons, why not have an automatic EM creation programme creating new EMs every time the total processing power is increased and allotting a subsistence level of hardware resources to each EM (having one 1000x EM prevents 999 other EMs from existing)? I want to keep hammering the point that Hanson is willing to throw many human rights under the bus of utility but the "right to collect way more crap than you need to be happy" (and only for a privileged few, who are often not even the most productive members of society, at that) is somehow preserved. Shouldn't he be old enough to know that you can't buy love, happiness or dignity?
"The original consented to both; this is like saying when I go to sleep today I should be painlessly killed, because my tomorrow's self did not consent to waking up."No, the original could be chilling on a luxurious island paradise and choose for the copy to become an enslaved drone even though the original would never want to become an enslaved drone himself. The copy and the original are two different people, they share the same memories but they are not the same persons from the moment of the copying onwards.
"You're missing the point, which is that children never consent to being born, which was your entire objection in this specific part of the argument."
My point was that although children do not choose to be born society forces the parents to either take good care of their children or hand them over to someone who will. This is perfectly possible to implement in an EM society as well, but Hanson chose not to include it in his version. Even if he would you waould run into problems because EM population explosions could happen in miliseconds whereas in the real world people get the chance to slowly experience the growing effects of overpopulation and change their behavior accordingly, taking multiple generations if need be (while EM society could get stuck in destructive traditions because there is no natural death to make room for open minded new generations).
"Painless death is fine - after all, you want to force it onto them by banning the technology use that would give them the option to live in the first place."
You are equating getting killed with never having lived. It's just not the same thing, and if it was you'd be damned if you do and damned if you don't, so you might as well pick the choice that gives people the highest quality of living. Besides, I have yet to see anything acting on the idea: Robin Hanson still does not have 20 kids, still has not given up his worldly possessions or committed suicide so the Earth can support more people.
"No, it really isn't."
So you really think life in the Soviet Union in the 1960s was worse than being some drone in a machine forced to slave away at subsistence level for an eternity, and getting killed if you slip up once?
"You assume that inequality is the same thing as slavery, but it is not; slavery takes your rights away while inequality simply means you have less resources than others."
It doesn't matter in practice if quitting my job results in me being put before the firing squad or starving to death in the gutter. If I cannot effectively control the course of my life, no matter how hard I work, then I am a slave.
"Not all games are zero-sum!"
Are you implying the 1000x EMs would suddenly grow a conscience? They can grow the pie a thousand times, but they're not at all obliged to share any of that growth with the rest. There is no competition, no free market when some people can buy radically faster brains and eternal life, capitalism doesn't simply doesn't work anymore, perhaps communism won't either, but there already exist alternatives to both and undoubtedly more will be invented by future thinkers.
"The original consented to being copied, not to become a copy"
The original consented to both; this is like saying when I go to sleep today I should be painlessly killed, because my tomorrow's self did not consent to waking up.
This is your reasoning to prevent life of intelligent beings who want to live?
"Normal reproduction os limited (a woman can only have so many children)"
You're missing the point, which is that children never consent to being born, which was your entire objection in this specific part of the argument.
"...then they will likely face enslavement or death."
Painless death is fine - after all, you want to force it onto them by banning the technology use that would give them the option to live in the first place.
"
Radical communism is heaven compared to Hanson's EM future, unless you are one of those few 1000x EMs..."
No, it really isn't.
The intuition you are holding that power abuse can induce suffering is exactly the reason why I reject top-down authoritarian morality.
You assume that inequality is the same thing as slavery, but it is not; slavery takes your rights away while inequality simply means you have less resources than others.
Not all games are zero-sum!
"1) We know pretty well what the em would want, because it is a copy of the individual who consents just moments prior."The original consented to being copied, not to become a copy who has to start life all over again for him/herself with nothing but the ones and zeros on their back.
"2) This radical requirement of absolute autonomy has never and will never be implemented; normal reproduction is much more imposing on the offspring since they require years of development before becoming intellectually mature."
Normal reproduction os limited (a woman can only have so many children) and the government steps in when children are abused, at least in civilized countries. In Hanson's world there will be no welfare programs to help newly created EMs get on their feet, they will just be thrown into the deep, if they are unlucky enough that no one left them anything and no one wants to take care of them then they will likely face enslavement or death.
"You assume ems will be sold as property; I assume they will be self-owned persons."
Hanson talks about EMs created for specific tasks and even if there's no official slavery there will be unofficial slavery serving the 1000x CPU power EM elite.
"3) Monocultures are vulnerable in specific ways; they would have to diversify or cooperate with others or their power base crashes."
Why? There's no physical violence and they are always literally 999 steps ahead of everyone else, and they can keep buying more CPU power, a 1x EM will never be able to catch up to them.
"4) I concede the point about externalities and I have no good solution except to say that freedom doesn't have to be all-or-nothing, and top-down bans on individual freedom have historically never led to good outcomes (compare radical communism)."
Radical communism is heaven compared to Hanson's EM future, unless you are one of those few 1000x EMs (but it's a strawman to immediately compare any rules that protect the weak to radical communism). The level of suffering in Hanson's world is almost incomprehensible to us, radical communism may have looked like a big deal in the past, but it's nothing compared to what the future can bring, so we should be careful about trying to run away from it so fast that we invite something even worse. The danger is that we will underestimate the possibilities of the future, both negative (potential for suffering if we apply obsolete concepts to a future world) and positive (the many ways smart thinking and advanced technology could make the average life more pleasant if we allow our economic thinking to evolve with our technology, relatively tiny infractions on the personal freedoms of the elite who are making much more than what their efforts justify, like setting a maximum wage of $1 million, could free countless souls from need and want).
"The EM is a separate entity, no one has the right to decide for it."
1) We know pretty well what the em would want, because it is a copy of the individual who consents just moments prior.
2) This radical requirement of absolute autonomy has never and will never be implemented; normal reproduction is much more imposing on the offspring since they require years of development before becoming intellectually mature.
"Youshouldn't sell an EM, just like you shouldn't sell a child."
You assume ems will be sold as property; I assume they will be self-owned persons.
I also assume they can shut themselves down painlessly whenever they want (there could be laws about that)
If you assume slavery, the whole debate becomes about slavery, not economics + futurism.
Just fight slavery.
"You are also forgetting that after a few EMs have bought 1000x CPU power they are essentially unstoppable"
1) Monocultures are vulnerable in specific ways; they would have to diversify or cooperate with others or their power base crashes.
2) I concede the point about externalities and I have no good solution except to say that freedom doesn't have to be all-or-nothing, and top-down bans on individual freedom have historically never led to good outcomes (compare radical communism).
Yeah (got your reference about a certain person's interest in cryogenics), it's NIMBYism to the extreme. These people never put their money (or their lives) where their mouth is, they're like chickenhawks, but even worse, preaching massive suffering, as long as it's someone else suffering.
"A person consents to scan his brain, becoming the initial em."The EM is a separate entity, no one has the right to decide for it. You shouldn't sell an EM, just like you shouldn't sell a child.
"There is never a point where authoritarian-utilitarian Hanson decrees that they should live in misery because it "always beats never having existed"."
Yes there is such a point: the EMs are created by someone who can afford not to share in their misery, yet Hanson (and you) argues that there should be no limits placed on copying. Essentially that's the same as saying children are their parent's property and parents don't have to look after their own children, that having laws against child abuse is wrong. It's also BS from a utalitarian viewpoint because if everything is subservient to utility, including the quality of life of the copies, then why go out of your way for some mundane "freedom to copy"?
You are also forgetting that after a few EMs have bought 1000x CPU power they are essentially unstoppable (they are not even mortal), any EMs created after that point will never be able to compete with the 1000x EMs (they think 1000x faster than a normal EM), who will then use their power to buy another 1000x CPU power and so on until the same handful of EMs dominate all the others forever, they will be the only ones for which the market will be free, everyone else will be a slave in all but name. Holding on to capitalism in such circumstances epitomises obsolete thinking which Hanson says he's against.
All in all it's quite a dystopian future Hanson is showing us, and I don't think it'll ever come to pass without some serious, violent resistance because many humans, and AIs, would definitely prefer death over Hanson's future.