In an article titled “What are the Benefits of Mind Uploading?” Michael Anissimov lists seven benefits: If the early adopters don’t go crazy and/or use their newfound abilities to turn the world into a totalitarian dictatorship, … others will then follow. … Suppose that millions of people choose to go for it. Widespread uploading would have huge effects. …
Mitch, I think you're way over-complicating things. You are the perfect example of another super-smart guy that believes weird things - you are of course in good company at 'Less Wrong' ;)
The answer to consciousness is really very simple. Dennett (the importance of narratives), Hofstadler (the importance of analogies) and Tononi (the importance of information integration) all had peices of the puzzle, and if you just put together what they say, that wraps up consciousness.
It's physical after all, it's just a computation yes, basically
categorization (analogy formation) + narratives (representations of values) = sentience (expressed as a complexity measure).
It's just categorization applied to the formation of representations of values (narratives). It really is that simple.
Yudkowsky and co are of course, dead wrong about key fundamentals yes, but you are challenging the 'Less Wrong' folks on the wrong points ;) The MWI of quantum mechanics and the physical reduction of consciousness are probably correct.
Having a twin copy-sister in cyber-heaven still leaves the embodied original where it was. And you are the embodied original.
In any case, the whole idea that an upload, a brain-simulation running on a digital computer, would even be conscious, is surely just a primitive misconception, along with the whole philosophy that consciousness is a type of computation for which "substrate" doesn't matter.
According to this philosophy, the same thoughts and feelings would inhabit any physical system implementing a particular state machine, whether it was made of neurons, transistors, dominos, or birds pecking at each other. The event of one bird pecking at another (in a "bird-pecking computer") has a mystic "0" or "1" inhabiting it, which is part of a larger binary state, and that state is where the mind lives.
Sorry, it's ridiculously dualistic and phantasmal and I don't buy it for one moment. It is actually *more reasonable* to think that qualia are quanta, consciousness inheres in some union of quanta into a "super-quantum" (perhaps through entanglement?), and that we just don't understand the brain's workings at the appropriate level of resolution yet. If that amounts to saying that there is a soul and it's a bose condensate of microtubular anyons in the dendrites of the prefrontal cortex - or something similar - so be it.
Locating the self in a single persistent entity makes a lot more sense than to say it's a "program" which hovers over "computations" which can be arbitrarily discontinuous in space and time - which is where the computational philosophy of mind leads you. The latter is false to the actual experience of life, which is that you are some place specific, that time has continuity, that you have a history and that history is part of you. The initial rationale for the deconstructed self of contemporary fashionable materialism may have been science, but in transhumanist circles it's also getting lots of impetus from the gnostic hope of a safe imperishable existence as a platonic posthuman program. I interpret this as an early cultural pathology of the emerging era. It remains to be seen how much power it will acquire, or whether the very attempt to implement the uploading agenda will force the next level of true knowledge upon us.
Does uploading really need to be sold? Perhaps to the young and healthy it does. For the rest it should make perfect sense as the optimal means to obliterate discontents with existence arising from biological existence, not the least of which include vulnerability to chronic pain, disability, and death. Do highly conceptual, speculative negatives really detract from its appeal given our primary teleologies of survival and avoidance of pain?
Ems may spend less time in leisure than we, and less in fantasy VR than we do TV and video games.I disapprove of you giving TV and videogames as the specific examples of leisure. Since those activities are fairly low-status forms of leisure their loss seems less tragic.
What you should have said is "Ems may spend less time in leisure than we, and spend less quality time with their families and friends in a shared VR environment than we do spending quality time with our families and friends in the materials world. Sharing time with friends and family is one of the most important and high-status forms of leisure, so focusing on it underlines the tragedy and horror that would result if ems have to spend so much time working.
This also underlines why Robin's inference that ems will have happy lives because "poor folks do smile" is based on a faulty analogy. Poor people, and people in third world countries, may not have a lot of money, but at least they usually have enough spare time to spend with their family and friends. If ems work more than 80 hours a subjective week, as Robin conjectures, that won't be true for them. (Robin has also suggested that workaholics smile too, but that's generally because all that hard work is buying them lots of money and status. Workaholic ems' hard work will buy them nothing more than subsistence, so it's another bad analogy.)
@arch1
Evidently one thing it means to be an em (or an em’s original) is being profoundly vulnerable.I think that in emWorld we will need a giant police force dedicated to searching for hidden computers full of kidnapped and enslaved ems. Fortunately the very fact that we will have ems will make such a police force easy to obtain.
Robin has suggested that stealthy enslavement of ems could be avoided by reproducing ems until they are at subsistence wages, thereby making slavery unprofitable. That suggestion seems obviously mistaken to me. For one thing it is a cure worse than the disease. For another, even in a subsistence world it would still be necessary to search for ems that have been enslaved for non-monetary reasons, such as torture for sadism's sake.
In fact, driving em wages down to subsistence would probably make it even harder to rescue ems that are being tortured for someone's personal amusement. If people value life less than they should when it is common, as Robin suggests, a subsistence world might not have the collective will to set up a police force to rescue tortured ems. They'd be too busy caring for themselves. A far more robust solution would be to keep em wages high, then tax them to pay for an army of em cops.
Why do you assume that at some point the EMs will not realize that modifying themselves destroys their identity?
If we take it for granted that human identity is just neural patterns (I do not believe that, but I think most people reading this post do), EMs will be able to "preserve" their identity better (they will not forget things, their simulated neurons will not die outside of data accidents) and thus they will put far more value on their identity than us. And, since we really tend to value our identity a lot, I think that the whole "self-improving digital persons" will turn out to be rather infeasible, unless EMs make copies of themselves, enslave them and then force them to upgrade themselves. But then again, why would you risk making your slaves more powerful than you?
Robin, why do you assume that em's will remain human-like in all aspects except the few you call out -- easier copying, lower resource consumption.
Why would they not take on massive memory, computational power, or additional computational modules specializing in some area that we humans are not good at; or even rewrite some of their own architecture or drives?
These would benefit ems, so they would probably do them if they could, and then most of your analysis, which assumes that ems remain almost completely human-like, fails.
Advocacy seems common and worthwhile to me. We are a social species - able to aggregate information from multiple sources. Advocates and detractors do not need to be combined into the same indivudual.
I disagree with everything from both of you. Here are my own thoughts on the consequences of the "benefits" that Michael listed:
#1. Ems, since they are Ems, are copies of people. When you and I take jobs we aren't doing it for the express purpose of growing the economy. Ems won't either. Although a concerted effort by Ems could result in tremendous growth, there is every reason to believe that Ems will not be enslaved for this purpose and will pursue the same desires as the people they were copied from, albeit more effectively. You could also argue that lots of Ems means fewer resources are required to produce the goods we need, resulting in higher unemployment and a larger divide between the wealthy and poor. Also, faster brains means the ability to make mistakes faster.
#2. Smarter does necessarily mean better. There is no correlation between intelligence and honesty. If you start making Ems smarter, you're going to create some super-villains. The super-heroes may fight and win, but not until the damage is done.
#3. Happiness is the enemy of progress. If I'm happy I have no incentive to improve my situation. While the level of happiness can be regulated, who is going to regulate it? Ems will find ways to take happiness (or adrenaline or dopamine) like a drug, to work around any regulation, and happily descend into oblivion.
#4. Nature recovers and fights back! We enjoy our superiorty and sequester ourselves to allow the environment to recover, allowing our ability to deal with the environment to decay. Earthquakes, tsunamis, tornados, bears, wolves, wasps, and ants infiltrate our infrastructure causing massive loss of Em life.
#5. Ems become VR addicted, contributing to the economic disaster from #1. Loss of grasp on reality allows #4 to occur.
#6. Closer connections cause marriages to fail at unprecedented levels. Em wars begin once everyone knows what we really think of them. Unscrupulous Ems figure out how to fake emotional responses and use it to take advantage of others.
#7. Ems consume all available physical resources. Humans start culling Ems so "natural humans" have resources to live. After the Ems win and some time passes, all resources are again consumed. Em black markets are created. Ems war with each other.
I've never been much of a believer in Terminator or Matrix-style dystopian futures, but thinking through the consequences of preserving the values and desires of people in Ems reminds me that humans have a long history of killing everything that can be killed, on massive scales, including each other. Why would Ems be any different?
Echoing Dremora's 2nd paragraph. I think that Robin's list of downsides doesn't sufficiently convey the exposure of ems (and to a lesser but still huge extent their originals) to a spectrum of "PII violations" which would make all currently prevalent concepts of identity theft seem trivial by comparison.
Evidently one thing it means to be an em (or an em's original) is being profoundly vulnerable.
Space based infrastructure should be both safe from ecological damage and less damaging to the ecology. Since all we need is to support computers in space, that's a heck of a lot easier than generating life support for humans. Cheap solar energy and radiant cooling make it far more ideal for computing than the surface of the earth. I recommend reading Keith Lofstrom's Server Sky proposal, where he discusses one possible approach to this.
Downsides 3 and 4 are not necessarily downsides. 3 could be a compensation for 2 and 5, and 4 is only bad if you think nature is intrinsically good in a way that cant be replaced in silica, which is doubtful.
An expansion on 6: It is easier to deceive, enslave or torture ems since their mind's architecture may be open to manipulation by the authorities. This can include reality fragmentation (uninformed Matrix inhabitants), even though that may conflict with intellectual worker competence. Ems will experience far more horrible torture in warfare or cyberterrorism than biological humans, and for longer subjective times.
A final disadvantage: The same resources could support far more ems than humans. While this is an advantage if their lives are great, it is a horrifying prospect if their lives are terrible.
There is no spoon.
Mitch, I think you're way over-complicating things. You are the perfect example of another super-smart guy that believes weird things - you are of course in good company at 'Less Wrong' ;)
The answer to consciousness is really very simple. Dennett (the importance of narratives), Hofstadler (the importance of analogies) and Tononi (the importance of information integration) all had peices of the puzzle, and if you just put together what they say, that wraps up consciousness.
It's physical after all, it's just a computation yes, basically
categorization (analogy formation) + narratives (representations of values) = sentience (expressed as a complexity measure).
It's just categorization applied to the formation of representations of values (narratives). It really is that simple.
Yudkowsky and co are of course, dead wrong about key fundamentals yes, but you are challenging the 'Less Wrong' folks on the wrong points ;) The MWI of quantum mechanics and the physical reduction of consciousness are probably correct.
Having a twin copy-sister in cyber-heaven still leaves the embodied original where it was. And you are the embodied original.
In any case, the whole idea that an upload, a brain-simulation running on a digital computer, would even be conscious, is surely just a primitive misconception, along with the whole philosophy that consciousness is a type of computation for which "substrate" doesn't matter.
According to this philosophy, the same thoughts and feelings would inhabit any physical system implementing a particular state machine, whether it was made of neurons, transistors, dominos, or birds pecking at each other. The event of one bird pecking at another (in a "bird-pecking computer") has a mystic "0" or "1" inhabiting it, which is part of a larger binary state, and that state is where the mind lives.
Sorry, it's ridiculously dualistic and phantasmal and I don't buy it for one moment. It is actually *more reasonable* to think that qualia are quanta, consciousness inheres in some union of quanta into a "super-quantum" (perhaps through entanglement?), and that we just don't understand the brain's workings at the appropriate level of resolution yet. If that amounts to saying that there is a soul and it's a bose condensate of microtubular anyons in the dendrites of the prefrontal cortex - or something similar - so be it.
Locating the self in a single persistent entity makes a lot more sense than to say it's a "program" which hovers over "computations" which can be arbitrarily discontinuous in space and time - which is where the computational philosophy of mind leads you. The latter is false to the actual experience of life, which is that you are some place specific, that time has continuity, that you have a history and that history is part of you. The initial rationale for the deconstructed self of contemporary fashionable materialism may have been science, but in transhumanist circles it's also getting lots of impetus from the gnostic hope of a safe imperishable existence as a platonic posthuman program. I interpret this as an early cultural pathology of the emerging era. It remains to be seen how much power it will acquire, or whether the very attempt to implement the uploading agenda will force the next level of true knowledge upon us.
Does uploading really need to be sold? Perhaps to the young and healthy it does. For the rest it should make perfect sense as the optimal means to obliterate discontents with existence arising from biological existence, not the least of which include vulnerability to chronic pain, disability, and death. Do highly conceptual, speculative negatives really detract from its appeal given our primary teleologies of survival and avoidance of pain?
Ems may spend less time in leisure than we, and less in fantasy VR than we do TV and video games.I disapprove of you giving TV and videogames as the specific examples of leisure. Since those activities are fairly low-status forms of leisure their loss seems less tragic.
What you should have said is "Ems may spend less time in leisure than we, and spend less quality time with their families and friends in a shared VR environment than we do spending quality time with our families and friends in the materials world. Sharing time with friends and family is one of the most important and high-status forms of leisure, so focusing on it underlines the tragedy and horror that would result if ems have to spend so much time working.
This also underlines why Robin's inference that ems will have happy lives because "poor folks do smile" is based on a faulty analogy. Poor people, and people in third world countries, may not have a lot of money, but at least they usually have enough spare time to spend with their family and friends. If ems work more than 80 hours a subjective week, as Robin conjectures, that won't be true for them. (Robin has also suggested that workaholics smile too, but that's generally because all that hard work is buying them lots of money and status. Workaholic ems' hard work will buy them nothing more than subsistence, so it's another bad analogy.)
@arch1
Evidently one thing it means to be an em (or an em’s original) is being profoundly vulnerable.I think that in emWorld we will need a giant police force dedicated to searching for hidden computers full of kidnapped and enslaved ems. Fortunately the very fact that we will have ems will make such a police force easy to obtain.
Robin has suggested that stealthy enslavement of ems could be avoided by reproducing ems until they are at subsistence wages, thereby making slavery unprofitable. That suggestion seems obviously mistaken to me. For one thing it is a cure worse than the disease. For another, even in a subsistence world it would still be necessary to search for ems that have been enslaved for non-monetary reasons, such as torture for sadism's sake.
In fact, driving em wages down to subsistence would probably make it even harder to rescue ems that are being tortured for someone's personal amusement. If people value life less than they should when it is common, as Robin suggests, a subsistence world might not have the collective will to set up a police force to rescue tortured ems. They'd be too busy caring for themselves. A far more robust solution would be to keep em wages high, then tax them to pay for an army of em cops.
Why do you assume that at some point the EMs will not realize that modifying themselves destroys their identity?
If we take it for granted that human identity is just neural patterns (I do not believe that, but I think most people reading this post do), EMs will be able to "preserve" their identity better (they will not forget things, their simulated neurons will not die outside of data accidents) and thus they will put far more value on their identity than us. And, since we really tend to value our identity a lot, I think that the whole "self-improving digital persons" will turn out to be rather infeasible, unless EMs make copies of themselves, enslave them and then force them to upgrade themselves. But then again, why would you risk making your slaves more powerful than you?
Robin, why do you assume that em's will remain human-like in all aspects except the few you call out -- easier copying, lower resource consumption.
Why would they not take on massive memory, computational power, or additional computational modules specializing in some area that we humans are not good at; or even rewrite some of their own architecture or drives?
These would benefit ems, so they would probably do them if they could, and then most of your analysis, which assumes that ems remain almost completely human-like, fails.
Advocacy seems common and worthwhile to me. We are a social species - able to aggregate information from multiple sources. Advocates and detractors do not need to be combined into the same indivudual.
I disagree with everything from both of you. Here are my own thoughts on the consequences of the "benefits" that Michael listed:
#1. Ems, since they are Ems, are copies of people. When you and I take jobs we aren't doing it for the express purpose of growing the economy. Ems won't either. Although a concerted effort by Ems could result in tremendous growth, there is every reason to believe that Ems will not be enslaved for this purpose and will pursue the same desires as the people they were copied from, albeit more effectively. You could also argue that lots of Ems means fewer resources are required to produce the goods we need, resulting in higher unemployment and a larger divide between the wealthy and poor. Also, faster brains means the ability to make mistakes faster.
#2. Smarter does necessarily mean better. There is no correlation between intelligence and honesty. If you start making Ems smarter, you're going to create some super-villains. The super-heroes may fight and win, but not until the damage is done.
#3. Happiness is the enemy of progress. If I'm happy I have no incentive to improve my situation. While the level of happiness can be regulated, who is going to regulate it? Ems will find ways to take happiness (or adrenaline or dopamine) like a drug, to work around any regulation, and happily descend into oblivion.
#4. Nature recovers and fights back! We enjoy our superiorty and sequester ourselves to allow the environment to recover, allowing our ability to deal with the environment to decay. Earthquakes, tsunamis, tornados, bears, wolves, wasps, and ants infiltrate our infrastructure causing massive loss of Em life.
#5. Ems become VR addicted, contributing to the economic disaster from #1. Loss of grasp on reality allows #4 to occur.
#6. Closer connections cause marriages to fail at unprecedented levels. Em wars begin once everyone knows what we really think of them. Unscrupulous Ems figure out how to fake emotional responses and use it to take advantage of others.
#7. Ems consume all available physical resources. Humans start culling Ems so "natural humans" have resources to live. After the Ems win and some time passes, all resources are again consumed. Em black markets are created. Ems war with each other.
I've never been much of a believer in Terminator or Matrix-style dystopian futures, but thinking through the consequences of preserving the values and desires of people in Ems reminds me that humans have a long history of killing everything that can be killed, on massive scales, including each other. Why would Ems be any different?
Echoing Dremora's 2nd paragraph. I think that Robin's list of downsides doesn't sufficiently convey the exposure of ems (and to a lesser but still huge extent their originals) to a spectrum of "PII violations" which would make all currently prevalent concepts of identity theft seem trivial by comparison.
Evidently one thing it means to be an em (or an em's original) is being profoundly vulnerable.
Space based infrastructure should be both safe from ecological damage and less damaging to the ecology. Since all we need is to support computers in space, that's a heck of a lot easier than generating life support for humans. Cheap solar energy and radiant cooling make it far more ideal for computing than the surface of the earth. I recommend reading Keith Lofstrom's Server Sky proposal, where he discusses one possible approach to this.
Downsides 3 and 4 are not necessarily downsides. 3 could be a compensation for 2 and 5, and 4 is only bad if you think nature is intrinsically good in a way that cant be replaced in silica, which is doubtful.
An expansion on 6: It is easier to deceive, enslave or torture ems since their mind's architecture may be open to manipulation by the authorities. This can include reality fragmentation (uninformed Matrix inhabitants), even though that may conflict with intellectual worker competence. Ems will experience far more horrible torture in warfare or cyberterrorism than biological humans, and for longer subjective times.
A final disadvantage: The same resources could support far more ems than humans. While this is an advantage if their lives are great, it is a horrifying prospect if their lives are terrible.
What infrastructure do you have in mind?
I disagree with your point 4. Ems will still depend on infrastructure which could be disrupted by ecological damage.
It’s just one article. I’m interested in considering the downsides of another article.
It's just one article. I'm interested in considering the downsides of another article.