Dvorsky Em Interview

The always thoughtful George Dvorsky just posted an interview with me on ems at io9. Dvorsky calls my scenario “Dystopian”:

These days, people worry about robots stealing our jobs. But maybe we should be more concerned about massive populations of computerized human brains. … Disturbingly, Hanson predicts that human servants could become status symbols among the rich.

“But to survive, most humans would need assets other than their ability to work,” Hanson told io9. “Like stocks, bonds, real estate, or patents. Humans without such assets may or may not get charity from other humans. Em charity to humans may focus on offering humans a chance to become ems.”

Disturbing, no? Hanson made it clear to me that his analysis is designed to forecast likely outcomes, and not those outcomes we currently desire. …

“In virtual reality, ems need never experience physical pain, hunger, disease, or grit,” says Hanson. …

Given these near-Dystopian scenarios, I asked Hanson what would motivate ems and why they would choose to cooperate with traditional power structures. …

As if a lot of this isn’t already grim enough, ems will have to deal with a host of other concerns. For example, a stolen copy of an em mind could be tortured to extract secrets, or enslaved to compete with the original. … Regardless, the future that Hanson describes is a far cry from what many of us are hoping for. Living as a stream of 1’s and 0’s may have its benefits — but a digital utopia has never seemed more elusive.

I wish Dvorsky would made it clearer what parts of this scenario he most dislikes. Then we could talk more productively about how to make it better. Perhaps I should have emphasized that with investments doubling monthly, humans might quickly turn even a small initial investment into a king’s ransom.

GD Star Rating
Tagged as:
Trackback URL:
  • Joe Teicher

    I think I know what is dystopian about your em scenario. You seem to think of ems as people. You seem to care about their well being. Many people would think of ems more like software, and care about their well being only as much as they care about the well being of microsoft word. To someone like that, your future looks like a place where many or most resources are consumed by computers for no one’s benefit. Humans are left with whatever scraps the computers leave for them. Seems like the future in “The Matrix” except without the matrix. Just a bunch of sweaty ravers in a cave somewhere in a world controlled by computers. That is less appealing than a scenario where humans keep control of their technology and are able to use it for their own benefit.

    • Robin Hanson

      But if the machines didn’t exist, humans would be a lot less rich. The machines create wealth, and humans owning a small fraction of that makes them richer.

      • IMASBA

        The EMs create more wealth but they keep it all for themselves because your scenario is laissez-faire capitalism and humans cannot compete with EMs. There will be more total wealth in the world, but humans won’t see much of it and neither will the typical EM. Humans could easily have less wealth than they do now (perhaps not even enough to survive) and you yourself have indicated that most EMs will exist at subsistence level and many will have very short lives. Humans cannot be offered to extend their existence by becoming EMs because the EM will be a copy (the human dies). Wealthy EMs can take everything away from poor EMs (they can charge to see a virtual sky or even to feel virtual ground under virtual feet). Faster EMs will get exponentially richer and faster, leading to an aristocracy.

        I’d say all of that sounds pretty dystopian and I doubt it’s a “bleak but realistic” future. There’s just no way the poor majority of EMs will put up with it for long. Revolution is inevitable and that’s assuming the laissez-faire system will come into existence at all (the first EMs will descend from future humans who might not like laissez-faire capitalism at all).

      • Robin Hanson

        If humans start out owning almost everything in the em world, then their fraction of wealth would slowly fall due to agency problems, but then still own a whole lot. If the ems “revolt” and take away all the human’s wealth, they’ll still be individually poor. There isn’t a revolution that can take a world of poor ems and make those ems rich.

      • IMASBA

        EMs can think faster, have a photographic memory, have a larger working memory, are better at math, they don’t require time to sleep, eat or bathe. How could humans possibly compete for a prolonged amount of time, or even survive?

        I wasn’t talking about EMs revolting against humans (though that might be how the EM world comes to be in the first place if humans decide to treat EMs as slaves), but of poor EMs against rich EMs. The poor EMs can redistribute the wealth of the rich EMs (and there will be a lot to go around because income inequality will be tremendously high owing to the differences in clockspeed, memory, etc… between rich and poor EMs). They can also restrict the creation of new EMs, to take collective advantage of economic growth.

      • Robin Hanson

        In a world of subsistence wages, redistributing from the rich still leaves wages near subsistence. It may be possible to pass and enforce draconian population control laws, but those hardly count as “revolutions.”

      • IMASBA

        “In a world of subsistence wages, redistributing from the rich still leaves wages near subsistence.”

        That really depends on the inequality. Who says an elite of the fastest EMs won’t receive half the total income or something like that? A doubling of income would surely be noticeable even if the original income was at subsistence level.

        Passing the draconian population laws would necessitate a power grab by the masses (who suffer from ever the effects of an increasing population) from a rich elite (who apparently profit from an increasing population, otherwise they would have passed draconian population laws). If a power grab doesn’t count as a revolution then nothing does.

      • Joe Teicher

        I guess I don’t understand why ems are necessary. Can’t you imagine a scenario where computer software continues to advance but never becomes a separate class of living beings that consume resources for their own benefit? In that scenario, we could have technology that is just as advanced as in the em scenario, but all the resources would be there for humans to consume. That definitely seems better to me. Your scenario rests on ems being able to do things that other software can’t do, but I don’t understand why that is likely. Software can already do a lot, without anywhere near the processing power needed to run an em.

      • Robin Hanson

        At the time ems become possible, I predict software isn’t good enough to do all jobs cheaply. If so, then it is a choice between using or not using the ems. Note that it is possible to imagine all the ems as slaves, and so not using any more resources than would software.

      • Joe Teicher

        I definitely think the “slave em” scenario is preferable to the “free em” scenario. The main problem with slave ems is the bad pr associated with the word slave. If slave ems were just called “advanced software” I think that would go over much better than trying to create a whole new class of people.

      • IMASBA

        If EMs are sentient beings (or appear to be sentient with the same certainty we have that other humans are sentient) then it doesn’t matter what euphemism for forced, unpaid labor you use, it’s still the same as human slavery.

        I do not share Robin’s certainty that EMs cannot possibly be replaced by non-sentient software, so slavery may be avoided after all.

      • MarkBahner

        What jobs do you see as not being doable by software at the time ems become possible?

      • Most of the jobs humans do today.

      • MarkBahner

        You think, “Most of the jobs humans do today” aren’t doable by software until ems become possible? That’s very surprising. I’ve just posted a table of the 15 most common jobs in the U.S. on my blog:

        It looks to me that almost all of them are vulnerable to software that’s far below the level of ems. For example, do you think cashiers need ems-level AI? Tractor-trailer drivers? Janitors?

      • Brent Dill

        Note: Janitors are listed twice on your chart.

      • MarkBahner

        #15 has been corrected to “elementary school teachers”…1.5 million of them, with an average annual salary of $53,000. I think they’re also vulnerable to artificial intelligence software that is well below ems-level. As I note in my updated blog post, perhaps the AI software will teach the children, while the cashiers who have lost jobs to AI software will come in to provide an adult in the room while the kids are learning from the AIs.

  • Doug

    “I should have emphasized that with investments doubling monthly, humans might quickly turn even a small initial investment into a king’s ransom.”

    Some humans early to invest in the EM economy will grow tremendously rich, but I don’t expect all of them will. If economic growth picks up by an order of magnitude then it’s inevitable that financial market volatility will also increase tremendously. More economic developments in compressed time means that a higher density of information will move asset prices at faster rates.

    Most investors set their risk tolerance and asset allocations based on historical financial returns. At the dawn of the EM era you’d expect a disconnect between past and future returns of financial markets. Many investors will be over Kelly-sized, and consequently the (unexpectedly high) volatility will wipe their wealth out before the super-high returns make them wealthy.

    • Robin Hanson

      The standard safe investments of index funds would also insure reasonably against risks in this transition. Among those who take more risks, some will lose badly. Not sure why one would expect it to be any other way.

      • Doug

        True, indexers would probably weather the transition smoother, but that investment strategy isn’t without its own problems in a sudden EM dawn. Any indexer still needs to choose a rebalancing strategy. The more frequently one rebalances the closer one tracks the hypothetical market portfolio, but the higher transaction costs.

        A sudden regime change would mean that most indexers would be historically biased towards rebalancing much too infrequently. For example the S&P 500 recomposes its index constituent weights on a quarterly basis. If economic activity sped up by a factor of 400 this would be like buying an index of stocks and just holding them for 100 years.

        Needless to say the performance of “slow indexing” is much worse than what we’re used to with standard indexing. (Remember the Simpsons episode where Mr. Burns asks how his investment in Confederate Slave Holdings is doing). Estimating how much worse is difficult. Ultimately all of this depends on just how quick the EM transition is. If it’s not so sudden most indexers will probably at least somewhat shift rebalancing to make up for it.

      • Doug

        The broader point I’m trying to make is that during abrupt economic transition “safe investment” becomes more difficult and perilous. Compared to stable periods the historical record that people are using to guide their investment decisions is more likely to lead you astray.

        For example the farmer to industry transition significantly increased growth rates. Yet the wealth-holding classes of the farmer industry all didn’t fare well in the industrial era. For example most of the great Houses of the British landed gentry went bankrupt by the 20th century.

        That’s because many followed the historical record of the farming era, when allocating wealth to land was the most stable and safest form investment. The industrial revolution turned that on its head by collapsing agricultural prices. So those who followed previously safe investment practices found their wealth dissipated.

      • Robin Hanson

        Didn’t we already know that there would be economic disruptions from time to time, and that this would make it harder to maintain wealth over the long run?

      • Doug

        I agree. But I would quibble: when the economic doubling time is on the order of months what constitutes the “long run” becomes way shorter than what meatspace humans are currently used to.

  • blink

    Given the growth you describe, it seems that a philanthropist could set up a trust sufficient to ensure, say, all future flesh-and-blood humans can attain a standard of living equal to median income in the US today. (Ems may replicate rapidly at low cost, but flesh-and-blood humans cannot.) Do you think something like this is likely? Desirable?

    • Robin Hanson

      That would have to be a *very* rich philanthropist, at least to cover the early em period.

  • I think on the one hand it is mere addition paradox stuff. Humans aren’t comfortable with adding trillions (billions?) of lives to the world at drastically reduced utility for the average person (ems being people, in that analysis), even if em lives are worth living.

    I think another issue is that the situation commercializes the act of creating human life. Most human lives we create are cared for and nurtured for years before they enter the marketplace. Creating lives for expressly comercial purpose, possibly without any of the bonds of love which exist even in societies where children work, is a thing you have really only see when humans own humans. It raises questions of whether ems whose creators cared for them significantly less well than ours did will have the same standard of living, rights, and all that etc our progenitors worked to get us.

    Then too there is the fact that significant control over ems’ sensory environment might allow new extremes of oppression. Earlier we had assumed ems would have lives worth living, but imagine an em with a life very much not worth living who was simply denied the ability to kill himself. Man has not always been so nice to his fellows, and the lurid possibilities of man’s inhumanity to em sort of well into the mind, bidden by enumerable science fictions.

    Lastly, I think most people have a healthy skeptism about our ability to predict how a society will look after a major disruption like ems. Major uncertainty + a tendency to distrust the unknown = distrust of an em future.

    • Robin Hanson

      You know who people love even more than their children? THEMSELVES. A scenario were people spin off copies of themselves is a scenario in which those new creatures are very loved.

      • Possibly true, but one person who does not love their ems can create billions more offspring than one person who does not love their children. Have you seen Moon?

      • It is possible in our world for slave owners to force their slaves to have kids, who are then quickly separated from parents. It is also possible for ems to be slaves and forced to make copies against their will. But it is also possible to have worlds like ours where kids are only made when parents want them, and to have em worlds where em copies are only made when their originals want them. The possibility of bad thing happening conditional on slavery doesn’t argue against a scenario unless you can argue that slavery is more likely in that scenario.

      • IMASBA

        Children serve a clear purpose in real life because parents grow weak and die (sorry I had to write such a sentence so soon). Having children is also a costly process that takes a huge chunk of the parent’s lifespans.

        In an EM world all of that would change and the smart thing to do for the vast majority of EMs would be to enforce strict regulations against unsanctioned (not intended to replace losses) copying. This could very well become a societal norm (new EMs could even be created with this norm implanted on a subconscious level, like an instinct). The death penalty for unsanctioned creators would both scare off offenders and help resolve the population surplus problem caused by unsanctioned copying.

        This all sounds horrible to us but the EM masses know unsanctioned copying is the biggest threat to their world. They won’t give up their world just so a few narcissists can spam the universe with copies of themselves.

      • Slavery in fact does exist in our world, and slave owners have forced their slaves to reproduce. The difference is that slave owners who force their slaves to breed have to go through a period of years in which the children consume resources and produce nothing. An Em slave owner would not. That dead-weight period is especially troubling for ilicit slavers, since it increases detection costs significantly. In Em world ilicit slavers could avoid that period.

        There’s every reason to think that the marginal cost of producinf a working “copy” would be substantially lower for slave owners who own Ems than those who own humans.

  • Lord

    I have a lot of problem with investments doubling monthly. In the real world we can measure real returns in terms of real product and these will always be limited by real constraints on what we can do, so we will have a real world that grows not much faster than we do now. The virtual world not facing these constraints or facing them in limited ways such as computer hardware and power can grow much faster but growth in the virtual world translates to massive deflation relative to the real world. There will be some leakage across the interface as virtual goods are provided to the real world and help growth of it, reduce waste, and increase efficiency, allowing the real world to approach its physical barriers more closely, but massive growth in the virtual world translates to a falling significance in the real world, and faster growth there translates to faster obsolescence, diminishing returns. The greater the return, the more investment undertaken until it falls to something comparable to other returns. It does matter how much leakage there is, as we could make everyone a billionaire as long as we had the assurance none would spent it, and returns can be huge as long as there are very limited attempts at realizing them.

    • Robin Hanson

      I think if we only counted physical assets like computers, buildings, roads, etc, we’d still measure the economy to be doubling monthly or faster.

      • Lord

        Do you believe in abundant energy, so plentiful or efficient that it becomes truly cheap? That is the only way I can see that happening.