Rating Ems vs AIs

Everyone without exception believes his own native customs, and the religion he was brought up in, to be the best (Herodotus 440BC).

I’ve given about sixty talks so far on the subject of my book The Age of Em. A common response is to compare my scenario to one where instead of ems, it is non-emulation-based software that can first replace humans on most all jobs. While some want to argue about which tech may come first, most prefer to evaluate which tech they want to come first.

Most who compare to ems to non-em-AI seem to prefer the latter. Some say they are concerned because they see ems as having a lower quality of life than we do today (more on that below). But honestly I mostly hear about humans losing status. Even though both meat humans and ems can both be seen as our descendants, people identify more with meat as “us” and see ems as “them.” So they lament meat no longer being the top dog in-charge center-of-attention.

The two scenarios have many similarities. In both scenarios, meat humans must all retire, and robots take over managing the complex details of this new world, which humans are too slow, distant, and stupid to manage. The world economy can grow very fast, letting meat get collectively very rich, and which meat soon starves depends mostly on how well meat insures and shares among themselves. But it is hard to offer much assurance of long run stability, as the world can plausibly change so fast.

Ems, however, seem more threatening to status than other kinds of sophisticated capable machinery. You can more vividly imagine ems more clearly winning the traditional contests whereby humans compete for status, and then afterward acting superior, such as by laughing at meat humans. In contrast, other machines can be so alien that we may not be tempted to make status comparisons with them.

If, in contrast, your complaint about the em world is that ems have a lower quality of life, then you have to either care about something more like an average quality of life, or you have to argue that the em quality of life is below some sort of “zero”, i.e., the minimum required for a life to be worth living (or having existed). And this seems to me a hard case to make.

Oh I can see you thinking that em lives aren’t as good as yours; pretty much all cultures find ways to see their culture as superior. But unless you argue that em lives are much worse than the typical human life in history, then either you must say the typical human life was not worth living, or you must accept em lives as worth living. And if you claim that the main human lives that have been worth living are those in your culture, I’ll shake my head at your incredible cultural arrogance.

(Yes, some like Nick Bostrom in Superintelligence, focus on which scenario reduces existential risk. But even he at one point says “On balance, it looks like the risk of an AI transition would be reduced if whole brain emulation comes before AI,” and in the end he can’t seem to rank these choices.)

GD Star Rating
Tagged as: , ,
Trackback URL:
  • Lord

    If it is impoverished, it would be in the sensory interface to reality or their virtual reality. Would they have smell or taste, or simulated smell or taste? Why? Is it even meaningful to simulate a mind without simulating a world in which it exists or all the apparati to interact with reality?

    • Chuck

      Assuming that we ever learn how to fully and faithfully create a model for emulating a human brain, and have reached a level of technological progress that would make emulating a human brain worthwhile, we would likely have sufficient knowledge to alter that emulated mind to make its need for “quality of life” irrelevant. Sensory information would be limited to whatever minimum is required to keep the EM on task and at work.

      Thoughts and feelings deleterious to producing work output can simply be removed or routed around. Perhaps an overriding compulsion to stay on task can be installed to eke out a little efficiency. Reward for work well done, or punishment for sloppy work can be accomplished with a few keystrokes by the operator. If an EM becomes belligerent, the sim can likely be modified to route around the problem or the EM can be erased and restored from a previous “save state” like any other program. EMs could be the perfect worker.

      • Lord

        They probably wouldn’t have any motivation then, or only motivation to work on what they find interesting. I think the story may be wrong as it is predicated on copying being cheap while copying in terms of costs of this virtual world will be fantastically expensive and not at all an option for an em.

      • Chuck

        Ah, but you are falling into the trap that an EM is a person or at least a black box whose emotions are driven by internal processes and can only be influenced by outside actors. We know enough about the brain right now to know the gross structures where emotions like motivation originate. There is no reason to think that when we finally can emulate a fully functional human brain we will not also have learned enough through experimentation to be able to directly influence thoughts and emotions. In essence, what an EM will want to do more than anything else, will be dictated by the operator. Certainly, there will be some EMs that will not be so tightly controlled, as there might be something lost by removing a part of their psyche, but this is all just so much speculation right now. I’m sure we’ll figure it out in time.

    • Peter David Jones

      That seems to questions: whether a virtual reality will be provided for ens,; and whether they will have real qualia.

  • I’m about halfway through your book. Really enjoying it! Agree with you about the human status angle. In fact, I wish you had used the title “Futurism is not about the Future” as your title for this post

    In any case, if you have time, regarding “While some want to argue about which tech may come first, most prefer to evaluate which tech they want to come first.” In fact I am very interested in which comes first. I find it extremely unlikely ems would come first before AGI. But suspect I am missing something in your argument, as you’ve thought about this for a long time. Do you have any supplementary blog posts or links on this topic (what comes first AGI or ems) that add to what you argue in the book? Or does the book capture your argument on this?

    • Few people can write down all that they know or think. But the core here is comparing observed rates of progress in the two directions.

  • endril

    You mention two concerns as if they’re closely related: Ems reduce meat-human status, and ems have low quality of life. Are these really part of the same objection? They practically seem like opposites:

    -Oh no, the future is going to be bad for me and my friends, our interests sacrificed for the sake of these *other* people! They’ll take our jobs! The world’s culture will primarily be about them rather than us! I don’t want this future because it gives all the attention to ems and I don’t empathize with ems.

    -Oh no, the future is going to be bad for my precious friends, the ems! They live at subsistence level, they have work all the time, they have alien demands placed upon them by the cruel and greedy super-future-economy! I don’t want this future because it neglects ems and I empathize with ems.

    • Even if they point in opposite directions, they are both both considerations used to justify a low evaluation of the em scenario.

  • lump1

    I imagine that a great deal of the labor done by all those ems would involve writing good task-specific AI, which will do most of the work that the world needs doing. Running AI will be far more computationally efficient than running ems, so ems will certainly not be truck drivers, miners, law clerks, etc. In fact, the coming of ems might actually jump-start the golden age of AI, because many more able minds can be assigned to improving it, and because once we see the details of a running mind emulation, we might be able to reverse engineer some neuro tricks and fold them into AI routines for added flexibility.

    Many ems will spend their last days coding and training an AI that can do their job adequately, but only needs a millionth of the CPU power that they require. Because of these potential savings, and because of the increased capabilities of em-coded AI, there will be lots of pressure to replace most working ems with AIs. Many of the ems that remain will become AI supervisors/overseers, intervening when AI routines report uncertainty or unexpected novelty. Since they will effectively be the commanders of distributed armies of AIs, a good name for their job would be: daemon-master.

    • I imagine that a great deal of the labor done by all those ems would involve writing good task-specific AI, which will do most of the work that the world needs doing.

      Perhaps folks who now do this for a living can testify whether this is exciting or boring work.

  • Michael Vassar

    I think that like me, Bostrom greatly prefers ems first but things that’s very unlikely to be the default outcome, though possibly worth fighting for.

  • I’m not sure that consistency requires thinking that if ems have a lower quality of life than makes life worth living, that most humans historically have had lives not worth living too.

    You might reasonably argue that morally there is a higher standard for ems, because we meathumans would be directly responsible for creating them and directly responsible for the foreseeable consequences thereof. Past humans just sort of evolved and persisted, no one/no group was responsible for their kinds of lives as a type.

    Of course, humans bring other individual humans into being in a sense too. But in the past it was not reliably possible to prevent this through contraception, and now that you can it does seem incumbent upon would-be parents to prevent creating children who will likely lead bad lives, even lives not as bad as the average past human but that would be quite bad by present standards.

    • You might take a less-consequentialist position that even though our ancestors lives were worth living, and we don’t regret their existence, it is immoral for us to create similar creatures. Because it matters less that we do good than that we show ourselves to be virtuous people. I spit on that view, but yeah, you could hold it. 🙂

  • Hedonic Treader

    “…then either you must say the typical human life was not worth living…”

    Pretty much this. Life is suffering. The small pleasures are supposed to outweigh not only the small pains in aggregate, but all the deliberate torture and other severe suffering in addition. That never adds up.

    The only thing that could change this is some kind of hedonic enhancement combined with sophisticated pleasure wireheading. But no one finds that attractive, and the economic pressures will select against it.

  • free_agent

    Dunno… Predicting how this particular wave is going to break seems to me to be a lot harder than, say, predicting in 1950 what the economy of 2015 will look like. (Right now, I’m reading Vonnegut’s “Player Piano”, and while some of his elements of dystopia are spot-on, many of them are completely incorrect.) There are just too many contingencies that might dramatically change the outcome. Remember the parody biology final exam question, “Create life, using the materials provided. Estimate the differences in subsequent human culture if this form of life had developed 500 million years earlier, with special attention to the British parliamentary system. Prove your thesis.”

  • Marc Geddes

    GAME: Solve AGI to beat EMS



    “Abstract: Solution to The Hard Problem of Consciousness”

    The universe is separated into 3 levels of abstraction – mathematical (basement), physical (mid-level) and mental (top-level).
    These 3 levels are ‘dual’ descriptions of reality – each is universal in scope, but not complete. (There is only 1 reality, but 3 different ways of describing it) – a ‘triple-aspect property dualism’.
    The reason for the 3-level split is a break-down in reductionism. Reductionism is the idea that the high-level properties could in principle be completely predicted from lower-level ones, and that the different levels of description are entirely consistent. Whilst a very good approximation most of the time, reductionism is not completely true. There are complexity thresholds past which reductionism *breaks* and entirely new properties appear not predictable in principle from lower-levels, nor completely consistent with the lower-levels.
    Extra logical principles will be needed to explain how to connect higher—level emergent properties to lower-level ones. These principles can be defined as ‘laws of causality’.
    Consciousness (subjective experience) occurs whenever there is a ‘transmission’ of information across different levels of organization. To be precise, it occurs whenever a ‘causal relation’ is represented; this requires the 3-level split that exists in reality to be mirrored by computation.
    When you have a computation that encodes for 3 levels of abstraction, then infinite recursion can always be approximated by at most 3 levels of abstraction: object, meta- and meta-meta . A computation represents a ‘causal relation’ by performing an information transmission across these 3 levels of abstraction.
    This property of information processing is present to some degree in most things, indicating some degree of consciousness is present nearly everywhere (panpsychism). Consciousness is the ‘informational glue’ that is ‘stitching’ the different levels of organization together. It is this ‘reality stitching’ operation that is making the universe increasingly coherent, by reducing the inconsistencies between the different levels of organization in the universe.
    The process that generates subjective awareness is also what generates the flow of time, working according to the above-mentioned ‘laws of causality’’. So consciousness is ‘stitching’ reality together: it is the principle of consistency and cohesion that enables reductionism to work.
    The purpose of life is revealed: to truly understand all of reality as an integrated (coherent) whole. For it is precisely the computations in our brains that generate new coherent categories of thought about reality (concept formation) that make us conscious and perform the ‘reality stitching’ operations described above!

  • dat_bro06

    Calling out homo hypocritus might delight the choir here but I hope (and trust) that for the sake of your prognostications about the impending age of EMs that you haven’t assumed that meat humans will ‘suddenly see the light’ about any of this!. Won’t complete domination by virtual human beings be a very bitter pill for the species to swallow, given its 200k year reign atop the food chain?

    • When humans feel they are on top they will feel free to voice dislike of ems, but once ems are on top most humans will be a lot more cautious about dissing their “overloads”.

  • > “your complaint about the em world”

    I can only answer for myself.

    While the point is complex and nuanced, I think I can simplify:

    1. I would prefer not to be captured and enslaved.
    2. If I were an EM, it might be put into such a situation (one of enslavement, which I do not prefer). Or, I may be forced into some kind of power vacuum (war / political maneuver) where some em’s enslave others. I would be required to play-to-win, which sounds difficult.
    3. People may be able to copy me against my will, and I would care about those copies (as much as “I” care about “myself”). They could be tortured, perhaps indefinitely. Torture would not cost tyrants very much. Or they could just “kill” me by shutting me off.

    So, I care about my EM-self, whom I perceive as in a state of vulnerability (approaching mortal peril), or at the very least in a state of extreme risk.

    However, it goes on:

    4. The representation of people as data (ie, mathematically), is a kind of jarring reminder that “I” and “other humans” are, in fact, ””on average””, the same.

    With all four, I’m at a confused place — I don’t know how much I should care about ‘all other EMs’ (or the entire EM-infrastructure). However, it seems that there is a risk that I should be caring about them as much as I care about myself. And it seems the potential risk to them is very great!

    Hence the complaining! I am neither surprised that [1] others are complaining, nor that [2] they don’t really know how to articulate the risks “they” are being placed into, and instead just Shout Stuff (they are looking -signaling-style- for allies to help them fight this new threat, no doubt).

    • asdf


  • Marc Geddes

    Abstract: ‘Fundamental Metaphysics : Solutiions to the Hard Problems’

    Marc Geddes, Auckland New Zealand, 10th June,

    I will show that all the ‘hard problems’ of metaphysics
    involving the ultimate origins of mathematics, physics and consciousness and the relationships between them, can be solved by assuming that there are 3 fundamental properties of existence that can none-the-less still be considered to be reducible *in the limit*. Each *fundamental* property
    can *also* be considered to be a *composite* property composed of 2 of the other properties in combination; in this ontology reality ‘eats its own tail’ in a closed loop.

    The long-standing tension between non-physicalists on the
    one hand, and hard-core reductionists on the other hand is finally resolved, with both sides being proven to be ‘half right’, yet the final solution being much more subtle than either side imagined.

    Three informal ‘plain English’ equations give a sketch of
    the final solution. Let us call these ‘The Fundamental Equations of Metaphysics’.

    Eq. (1)
    Physics + Math = Mind

    (This says that conscious awareness is a combination of matter and information)

    Eq. (2)
    Math + Mind = Physics

    (This says that matter is a combination of information and conscious awareness)

    Eq. (3)
    Physics + Mind = Math

    (This says that information is a combination of matter and conscious awareness)

    Equation (1) tells you what ‘Consciousness’ (Mind) is.

    It says that the property of ‘consciousness’ can also be
    considered to be a *combination* of the other 2 properties; matter *and* information. Consciousness can be viewed as *both* a
    fundamental property *and* a composite property. This paradoxical sounding statement is resolved by granting that consciousness can be considered as *both* a single property
    not 100% reducible to any other properties, *and* a composite property reducible to the other 2 properties *in the limit*
    (i.e., you can give a computable *approximation* of consciousness to any desired degree of accuracy in terms of matter and information, but never achieve a 100% complete reductive account).

    The solution is similar for the other two properties of
    existence (Matter and Information).

    Equation (2) tells you what ‘Matter’ (Physics) is.

    It says that the property of ‘matter’ can be considered to
    be a *combination* of the other 2 other properties, information *and* consciousness. The new composite property (matter) is reducible to the other 2 properties *in the limit*

    Finally, Equation (3) tells you what ‘Information’ (Math)

    It says that the property of ‘information’ can be considered
    to be a *combination* of the other 2 properties, matter *and*
    consciousness. The new composite property (information) is reducible to the other 2 properties *in the limit*.

    The circle is complete and reality ‘eats its own tail’, with
    each of the 3 ‘fundamental’ properties of existence (consciousness, matter and information) being ‘reducible in the limit’ to 2 of the others.