The new movie True Grit is a well-done celebration of the central farmer value of self-control. Not that foragers didn’t use self-control, but farmers took it to a whole new level, to get folks to act in quite un-forager-like ways. George Will’s latest column echos that gritty farmer theme, with a standard conservative warning that wealth and modernity tempts us
Robin thinks that human emulations will be practical before a designed general artificial intelligence will be, and an emulated human will naturally be conscious and have (to some, currently unknown, extent) human emotions. Later they may be able to edit out particular capacities that are not desired for particular jobs. It may not be possible, on the other hand, to have an effective general intelligence that is not conscious and emotional - that is one thing that is not yet known either way.
So far, economic development of machines suggests that the most efficient forms for doing various tasks are NOT human forms. Why do you think that trend will stop, and even reverse, and that ems will be the most productive machines, as opposed to machines designed to do the various production jobs? I foresee the richness of robots we see in factories today, but smaller, and larger, and more autonomous. But why make them conscious or capable of sadness or anger?
"Today capitalism has a bipolar disorder, demanding self-controlled workers yet uninhibited shoppers. …"
What you are referring to is the Keynesian-style form of society - a style of central economically controlled pseudo-capitalism operating under key tenants of consumerism. This is not capitalism. The key economic view of the John Maynard Keynes School of thought is that spending, cheap credit (spread around by the banking sector through a low-interest discount window from the Fed) - and not capital structure and savings as promoted by those within the classical-liberal Friedrich von Hayek Austrian School, creates real market growth.
True Capitalism is the free market. The free market - that is, a condition of society in which all economic transactions are a result of voluntary choice without coercion by the state, awards those who are both self-controlled workers and savers. The "paradox of thrift" is a great fallacy.
Keynesian style pseudo-capitalism has the bi-polar disorder.
To put it another way: Human have the luck that they are currently in a Darwinian disequilibrium. Contraception doesn't kill lust. The industrial revolution gave us a demographic transition which reduced fecundity. We can enjoy the evolutionarily sub-optimal superstimuli that our technology makes possible. Fortunately, we have not been
truly enslaved by evolution, gradually turning into true fitness maximizers obsessed with outreproducing each other. But thankfully that's not what happened. Not here on Earth. At least not yet.The hard part is keeping this. How do we extend and stabilize the demographic transition so that neither fast-breeding human subgroups nor copyable ems force us back into the malthusian trap?
Thanks for the reply! The difference is that pigs aren't being bred for making choices that give maximum long-term fecundity. I'd guess that breeders select for traits that give the most economical meat. The pigs make no choices, so there is no pressure to select for any particular mental state of the pig. Yes, a happy pig should be compatible with pig breeders' goals.
By contrast, for ems, I'm assuming Prof. Hanson's The crack of a future dawn scenario:
Darwinian evolution of values should once again become a powerful force in human history. Most uploads should quickly come to value life even when life is hard or short, and wages should fall dramatically.Because the ems make choices, and these choices affect their differential fecundity, evolution should be expected to tune their motivations and psychological structure to value outcomes that maximize their fecundity - and in a stable population, these outcomes must mostly fail. (I am assuming that the unit of selection is the em - Shulman's paper is an interesting alternative, but I haven't thought through its consequences)
It's an interesting argument, but I don't think the conclusion follows. Assume that the world's population of pigs is at an economic saturation point (cost of raising another pig = value of pig). Pigs are creatures with goals, but from these two facts, does it follow that these goals are frustrated? Can't there be, despite the saturation, only happy pigs? I think yes. In fact, if someone engineered a happier pig, I would hope that for ethical reasons, all slaughtered pigs are replaced with these happier models. So it goes for ems.
I don't know if Prof. Hanson wrote with ironic humor, but it's hard to escape that the Em existence as he describes won't be much different in kind than current human existence.
-There are 6 billion of us. At least 3 billion of us functionally live an em life. A life not of leisure, a life not of unproductive dysfunction, but a worker's life as part of horde.
-We are observed by the institutions we work for through the lense of aggregrate statistics.
If so, ems should feel hunger, pain, desire, and deep frustration, because such things are useful to feel.More generally, if ems are subjected to sharp enough selection pressure then we should expect that their motivations will become optimized for replication fitness. Over the long term, the population of ems must saturate at some level that physical resources can sustain. Therefore, a stable population of ems with evolutionarily optimized motivations will mostly have their motivations frustrated. Whatever it is that they wind up wanting (effectively for replication, though it might not feel that way internally), bits, atoms, sunlight, whatever, will largely be what they don't get.
I concur with Hedonic Treader's point of view: If this is the future,there is no point in building it.
The future is now. In what way are such working animal as border collies,drug sniffing dogs and race horses fundamentally different ,other than their mechanics,from ems?
Full agreement here. If ems are controlled by societies of "meatbags", they could even be programmed to find deep satisfaction in serving them (us).
"If so, ems should feel hunger, pain, desire, and deep frustration, because such things are useful to feel." (OP of Robin Hanson)
I still hope that most of the sticks can be replaced by specifically designed carrots. The same logic could be applied to organic sentients as well: www.gradients.com
If the best we can hope for is the continuation and scaling-up of average darwinian misery, there is no reason to build such a future in the first place; we might just as well nuke the planet now. We've already had hundreds of millions of years of that net-negative paradigm, I fail to see why we should want more of it.
It's just a no-brainer (ha!) that even if ems begin as "brain scans rendered in software" we will almost immediately tweak the em capacities and motivational systems. We already have much experience with these tweaks in wet brains, but our tools - chemicals - are incredibly blunt. Once a brain is software, very precise tweaks will be trivial to make, instantly reversible, and immediate feedback will be available.
Don't think about ems as necessarily coming in generations. Even if our first ems are perfectly "natural", they can (and will) have their motivational system upgraded with trivial ease. To them it would feel like they swallowed a Prozac pill and suddenly find great joy in designing weapons or whatever. The disanalogy is that "software Prozac" will be easy to tune for very specific effects.
Actually, this might be a better analogy: Imagine neurosurgeons could perform quick lobotomies of atom-scale precision, get immediate feedback from their patients, instantly undo their changes, and have a control brain which is atom-for-atom identical to the brain they started with. How long would it take them to design a person with an optimized or a specifically targeted motivational set? Weeks, if not less. Now consider that whoever runs the em will have even more control over their cognition systems, because unlike lobotomies, the software engineers can add parts as well as remove them. This is why I think it's folly to speculate about multitudes of "people like us" rendered in software. There is no incentive for their creation and maintenance. We will from the start make improved people.
I'd go so far as to say that if I were to retire into em-hood after my biological systems started to fail, I would want to then rewire my pleasure and reward systems in substantial ways. I would, for example, wire triggers to simulate the effects of hitherto unimaginable recreational drugs. But when the recreation was over and it was time to toil, I would definitely want to make myself love and find deep meaning in the toil I do - like Sisyphus. Accomplishing this as an em would be easy. There is no reason to even make a toiling em who doesn't love it. Arguably, it would even be immoral.
You are primarily looking at the economic aspects of ems and hence are able to extrapolate all the way to malthusian subsistence. But human beings have almost never taken getting poor or lower status peacefully.
I'm not sure how you think that property rights like those in land and patents will continue to be respected in a world where people are approaching subsistence?
What are the incentives for governments or secure systems to respect property rights? Governments can, with ems, dissolve the physical people.
From the perspective of ems, what is your prescription against a new french revolution occuring ? Atleast the initial version will be very smilar to present day humans. They will need their leisure to work properly and in this leisure time, they will think thoughts of fairness and unfairness, of meatbags using huge multiples of their resources and living in luxury. Do you believe that in the long run, there is no difference to the scenario?
I can imagine that a desperate war of all against all might very well occur and result in the winner establishing a singleton.
They are not (at least initially) “designed” or “programmed”, though we may expect them to evolve as time goes on.
I would expect the "designer ems" to show up fairly quickly after we had the unaltered kind. Will these first generation, human-like ems remain competitive for long enough to be the real inheritors in the em revolution?
I think it's still difficult to say what life will be like for ems. How easy will it be to alter their systems of motivation, for example? If it's not that hard, a majority of them might be quite satisfied with constantly doing work that humans wouldn't like.
Robin thinks that human emulations will be practical before a designed general artificial intelligence will be, and an emulated human will naturally be conscious and have (to some, currently unknown, extent) human emotions. Later they may be able to edit out particular capacities that are not desired for particular jobs. It may not be possible, on the other hand, to have an effective general intelligence that is not conscious and emotional - that is one thing that is not yet known either way.
So far, economic development of machines suggests that the most efficient forms for doing various tasks are NOT human forms. Why do you think that trend will stop, and even reverse, and that ems will be the most productive machines, as opposed to machines designed to do the various production jobs? I foresee the richness of robots we see in factories today, but smaller, and larger, and more autonomous. But why make them conscious or capable of sadness or anger?
"Today capitalism has a bipolar disorder, demanding self-controlled workers yet uninhibited shoppers. …"
What you are referring to is the Keynesian-style form of society - a style of central economically controlled pseudo-capitalism operating under key tenants of consumerism. This is not capitalism. The key economic view of the John Maynard Keynes School of thought is that spending, cheap credit (spread around by the banking sector through a low-interest discount window from the Fed) - and not capital structure and savings as promoted by those within the classical-liberal Friedrich von Hayek Austrian School, creates real market growth.
True Capitalism is the free market. The free market - that is, a condition of society in which all economic transactions are a result of voluntary choice without coercion by the state, awards those who are both self-controlled workers and savers. The "paradox of thrift" is a great fallacy.
Keynesian style pseudo-capitalism has the bi-polar disorder.
- Nathan Wosnack
To put it another way: Human have the luck that they are currently in a Darwinian disequilibrium. Contraception doesn't kill lust. The industrial revolution gave us a demographic transition which reduced fecundity. We can enjoy the evolutionarily sub-optimal superstimuli that our technology makes possible. Fortunately, we have not been
truly enslaved by evolution, gradually turning into true fitness maximizers obsessed with outreproducing each other. But thankfully that's not what happened. Not here on Earth. At least not yet.The hard part is keeping this. How do we extend and stabilize the demographic transition so that neither fast-breeding human subgroups nor copyable ems force us back into the malthusian trap?
Thanks for the reply! The difference is that pigs aren't being bred for making choices that give maximum long-term fecundity. I'd guess that breeders select for traits that give the most economical meat. The pigs make no choices, so there is no pressure to select for any particular mental state of the pig. Yes, a happy pig should be compatible with pig breeders' goals.
By contrast, for ems, I'm assuming Prof. Hanson's The crack of a future dawn scenario:
Darwinian evolution of values should once again become a powerful force in human history. Most uploads should quickly come to value life even when life is hard or short, and wages should fall dramatically.Because the ems make choices, and these choices affect their differential fecundity, evolution should be expected to tune their motivations and psychological structure to value outcomes that maximize their fecundity - and in a stable population, these outcomes must mostly fail. (I am assuming that the unit of selection is the em - Shulman's paper is an interesting alternative, but I haven't thought through its consequences)
It's an interesting argument, but I don't think the conclusion follows. Assume that the world's population of pigs is at an economic saturation point (cost of raising another pig = value of pig). Pigs are creatures with goals, but from these two facts, does it follow that these goals are frustrated? Can't there be, despite the saturation, only happy pigs? I think yes. In fact, if someone engineered a happier pig, I would hope that for ethical reasons, all slaughtered pigs are replaced with these happier models. So it goes for ems.
Wei, we have talked about it.
I don't know if Prof. Hanson wrote with ironic humor, but it's hard to escape that the Em existence as he describes won't be much different in kind than current human existence.
-There are 6 billion of us. At least 3 billion of us functionally live an em life. A life not of leisure, a life not of unproductive dysfunction, but a worker's life as part of horde.
-We are observed by the institutions we work for through the lense of aggregrate statistics.
Robin, have you read and/or commented on Carl Shulman's recent paper Whole Brain Emulation and the Evolution ofSuperorganisms?
If so, ems should feel hunger, pain, desire, and deep frustration, because such things are useful to feel.More generally, if ems are subjected to sharp enough selection pressure then we should expect that their motivations will become optimized for replication fitness. Over the long term, the population of ems must saturate at some level that physical resources can sustain. Therefore, a stable population of ems with evolutionarily optimized motivations will mostly have their motivations frustrated. Whatever it is that they wind up wanting (effectively for replication, though it might not feel that way internally), bits, atoms, sunlight, whatever, will largely be what they don't get.
I concur with Hedonic Treader's point of view: If this is the future,there is no point in building it.
The future is now. In what way are such working animal as border collies,drug sniffing dogs and race horses fundamentally different ,other than their mechanics,from ems?
Full agreement here. If ems are controlled by societies of "meatbags", they could even be programmed to find deep satisfaction in serving them (us).
"If so, ems should feel hunger, pain, desire, and deep frustration, because such things are useful to feel." (OP of Robin Hanson)
I still hope that most of the sticks can be replaced by specifically designed carrots. The same logic could be applied to organic sentients as well: www.gradients.com
If the best we can hope for is the continuation and scaling-up of average darwinian misery, there is no reason to build such a future in the first place; we might just as well nuke the planet now. We've already had hundreds of millions of years of that net-negative paradigm, I fail to see why we should want more of it.
It's just a no-brainer (ha!) that even if ems begin as "brain scans rendered in software" we will almost immediately tweak the em capacities and motivational systems. We already have much experience with these tweaks in wet brains, but our tools - chemicals - are incredibly blunt. Once a brain is software, very precise tweaks will be trivial to make, instantly reversible, and immediate feedback will be available.
Don't think about ems as necessarily coming in generations. Even if our first ems are perfectly "natural", they can (and will) have their motivational system upgraded with trivial ease. To them it would feel like they swallowed a Prozac pill and suddenly find great joy in designing weapons or whatever. The disanalogy is that "software Prozac" will be easy to tune for very specific effects.
Actually, this might be a better analogy: Imagine neurosurgeons could perform quick lobotomies of atom-scale precision, get immediate feedback from their patients, instantly undo their changes, and have a control brain which is atom-for-atom identical to the brain they started with. How long would it take them to design a person with an optimized or a specifically targeted motivational set? Weeks, if not less. Now consider that whoever runs the em will have even more control over their cognition systems, because unlike lobotomies, the software engineers can add parts as well as remove them. This is why I think it's folly to speculate about multitudes of "people like us" rendered in software. There is no incentive for their creation and maintenance. We will from the start make improved people.
I'd go so far as to say that if I were to retire into em-hood after my biological systems started to fail, I would want to then rewire my pleasure and reward systems in substantial ways. I would, for example, wire triggers to simulate the effects of hitherto unimaginable recreational drugs. But when the recreation was over and it was time to toil, I would definitely want to make myself love and find deep meaning in the toil I do - like Sisyphus. Accomplishing this as an em would be easy. There is no reason to even make a toiling em who doesn't love it. Arguably, it would even be immoral.
Prof Hanson, A happy new year to you.
You are primarily looking at the economic aspects of ems and hence are able to extrapolate all the way to malthusian subsistence. But human beings have almost never taken getting poor or lower status peacefully.
I'm not sure how you think that property rights like those in land and patents will continue to be respected in a world where people are approaching subsistence?
What are the incentives for governments or secure systems to respect property rights? Governments can, with ems, dissolve the physical people.
From the perspective of ems, what is your prescription against a new french revolution occuring ? Atleast the initial version will be very smilar to present day humans. They will need their leisure to work properly and in this leisure time, they will think thoughts of fairness and unfairness, of meatbags using huge multiples of their resources and living in luxury. Do you believe that in the long run, there is no difference to the scenario?
I can imagine that a desperate war of all against all might very well occur and result in the winner establishing a singleton.
They are not (at least initially) “designed” or “programmed”, though we may expect them to evolve as time goes on.
I would expect the "designer ems" to show up fairly quickly after we had the unaltered kind. Will these first generation, human-like ems remain competitive for long enough to be the real inheritors in the em revolution?
I think it's still difficult to say what life will be like for ems. How easy will it be to alter their systems of motivation, for example? If it's not that hard, a majority of them might be quite satisfied with constantly doing work that humans wouldn't like.