Caplan on Age of Em

As many have noted, ours in an era of ideological polarization. On topics where there are strong emotions, we tend to gravitate to extremes, and are less interest in intermediate positions. Which is a problem for my book; while most see it as too weird, others mostly see it as not weird enough. A  tech futurism minority expects to soon see very rapid progress in artificial intelligence and machine learning, and so see brain emulations as too slow and inefficient compared to the super-intelligence they foresee. And the majority to whom that seems pretty crazy also seem brain emulations as similarly crazy; they don’t care much if ems seem a bit less crazy.

At least future tech enthusiasts who think my book not weird enough are willing to write reviews to say so. But those who think my book too weird mostly stay silent; I’ve heard privately of many who were going to cover the book before they fully realized what it is about. So I thank my college Bryan Caplan for being willing to say what others won’t, in his critical review. His review is long, with ten criticisms. This response will also be long, going point by point.

Six of his ten objections seem to be mainly about my language. (His review is indented, and often contains book quotes; my replies are not.)

1. Robin only pretends to dodge philosophy of mind. .. He tacitly accepts an extreme version of “Ems are just as human as you or me” – and builds the whole book on this assumption. The tell-tale sign: The Age of Em says vanishingly little about the lives of biological humans during the Age of Em! .. he seems so wedded to this philosophical (not social scientific) position that he can’t even feign agnosticism. What would feigning look like? Split the book evenly between discussion of the lives of biological humans during the Age of Em and ems during the Age of Em.

These are complaints about language and emphasis. On language, since I’m constantly applying human and social sciences to ems it would have been quite awkward to use any other than our usual language for describing people. It is hard to imagine a readable book where words like “people” are constantly replaced with phrases like “machines that act like humans but do remember I’m not making any philosophical claims here.” On emphasis, very little happens to biological humans during the em era, so equal emphasis would mean a very short book. That conflicts with my goal of showing just how much I can say about this scenario.

3. Robin has a bizarre definition of “marginalized” .. biological humans .. They’ll be outnumbered, and perform little “hands-on” work. But they’ll be fabulously wealthy and ultimately in charge.” .. 5. Robin’s conclusions only sound “taboo” because he’s using language strangely. .. Robots will “dominate” us no more than rank-and-file workers “dominate” shareholders.

As I mentioned in a previous post, many have reacted to talks I’ve given by complaining about humans no longer being at the center of action, even when they understand that biological humans could for a while own most of the em world, and thus direct how spare resources are spent. I used words like those people use to acknowledge their concerns.

4. Contrary to Robin’s suggestion, there’s near-zero correlation between income and conservatism. .. “subsistence farmers tend to have values more like those of poor/conservative people today.” .. Robin could say he’s defining “conservatism” in a technical or apolitical way. But when you’re writing for an audience, the author rightly bears the burden of highlighting non-standard usage.

In that context I had just cited studies on strong correlations between the culture and wealth of nations, and I had just explained in quite some detail the kind of “conservatism” I meant there. It is true that I didn’t mention explicitly there that the word “conservatism” is used in many different ways. But the book would be a lot more tedious if every time I introduced and used a term I explained the many other ways people have used that term.

7. Robin’s efforts to calm readers’ fear of the future consistently backfire. Example:

Readers of this book may find near subsistence wages to be a strange and perhaps scary prospect. So it is worth remembering that such wages in effect applied to almost all animals who ever lived, to almost all humans before a few hundred years ago, and for a billion humans still today. Historically, it is by far the usual case.

Imagine a middle-class American’s child is destined to earn a subsistence wage. Would it make the parent feel better to hear, “No big deal, your child will face the same fate as almost every animal who ever lived, almost all humans before a few hundred years ago, and a billion humans today”? No, even worse!

Saying something is “worth remembering” just means that it may change how you think about that thing; it is not the same as saying “don’t worry.” It isn’t my job to make readers like the age of em, but it is my job to make sure they keep important considerations in mind.

10. Robin’s argument against the Terminator scenario is much weaker than it looks. His words:

A reasonable hope is that ordinary humans become the retirees of this new world. .. ems may be reluctant to expropriate or exterminate ordinary humans if ems rely on the same or closely interconnected legal, financial, and political systems as humans, and if ems retain many direct social ties to ordinary humans.

The problem: As Robin explains, in one human year, ems experience millennia. So even if each generation of ems only has a .5% chance of expropriating humanity, the chance of expropriation per human year is around 40%.

Bryan misreads me as trying to offer more reassurance than I can. I was clear that even if humans survive the year or two that comprises the age of em, I can say little about what might happen after that. “A reasonable hope” is quite different from “don’t worry.” I would be remiss if I didn’t at least point readers in the direction of a reasonable hope, even if I can offer few guarantees.

Two of Bryan’s objections seem to make opposite points on human constancy:

2. Robin exaggerates how dramatically humans have changed over time: .. `Historical fiction misleads you, showing your ancestors as more modern than they were.’ .. And contrary to Robin, I see a largely constant human nature. Characters in Shakespeare seem as credible to me.

This is a tiny point in the introduction, and little else in the book depends on it. Bryan has read enormous amounts of history, making him an exception mislead far less by historical fiction than are most people. My warning is to them, not him.

6. Robin greatly overstates the difference between his “em scenario” and the “generalized AI scenario.” How so? The Age of Em makes numerous arguments by analogy: Since humans typically do X in situation Y, and ems are copies of humans, ems will also typically do X in situation Y. But he also keeps telling us that only a tiny hand-picked sub-sample of humans will be copied. The obvious question: Why wouldn’t ems largely be copies of the most “robot-like” humans – humble workaholics with minimal personal life, content to selflessly and uncomplainingly serve their employers? This in turn implies that most of Robin’s “detail” is roughly the opposite of what would really happen.”

We have already seen a lot of historical variation in both how much workers submit at work, and in how many hours they work. As I discuss in the book, from around 1820 to 1850 in the U.S., France, and Germany, men worked at jobs an average of 68 to 75 hours per week. And workers in rich nations today accept far more explicit dominance and ranking at work than most of foragers and farmers would have accepted. Even so, there is plenty of commonality between these past and current humans, enough to justify our usual practice of using analogy with humans today to understand past humans. Similarly, even if ems are somewhat more submissive and workaholic than workers today, there should still be enough commonality for analogies to be useful in understanding ems. And as I explained in my response to Scott Alexander, extreme submission and workaholic scenarios seem implausible.

Bryan’s last two objections, on economics, are the ones I take most seriously.

8. Robin greatly overstates the quality of life for ems. .. Why wouldn’t ems’ creators use the threat of `physical hunger, exhaustion, pain, sickness, grime, hard labor, or sudden unexpected death’ to motivate the ems? Robin elsewhere talks about `torturing’ ems, so why not?” .. Modern systems of slave labor – see Stalinist Russia and Nazi Germany – used pain freely, because the penalty for quitting was death.

I was careful not to claim that ems would not be slaves. I just suggested that we didn’t have good reasons to expect most being slaves. Most people in history haven’t been slaves, even when competition has been strong. There are still some slaves in the world today, and they aren’t known for being spectacularly productive workers due to frequent use of torture and pain. Nor were Stalinist Russia and Nazi Germany slaves known for stellar productivity. We have a larger literature on motivating workers, and threats of pain only seem to be useful for the most routine and physical of labor. When you seek workers to be creative, think carefully, take the initiative, or persuade and inspire others, you mostly seek other motivations. Our best analogy for ems should be the few hundred most productive people in the world today, and threats of pain are not remotely what motivate them. Who thinks torture would make them more productive overall in the long run?

9. Robin’s arguments for his single craziest claim – global GDP will double every “month, week, day, or even faster” – are astoundingly weak. .. In the real world, however, there are literally hundreds of bottlenecks that radically retard this kind of growth. Politically, something as simple as zoning could do the trick. .. the most favorable political environments on earth still have plenty of regulatory hurdles .. we should expect bottlenecks for key natural resources, location, and so on. .. This alleged “concrete clue” is nothing compared to the bona fide “concrete clue” that almost all fantastic claims are false. And the idea that the global economy will start doubling on a monthly basis is fantastically fantastic. This has to be the least Bayesian part of the book: We start with a claim with a near-zero prior probability, make a couple of flimsy arguments, and somehow end up with a high posterior probability.

One could have similarly argued that fundamental growth bottlenecks must prevent the previous observed huge jumps in growth rates, such as from foraging to farming, or farming to industry. And plausibly related obstacles did prevent those eras from starting as soon as they might have. But eventually obstacles were overcome. No doubt our current economy tolerates many delays that would have to be cut to enable much faster growth, and the em economy won’t start as early as it might because of regulatory and other delays. My book is mainly about what happens once those obstacles are overcome. Does Bryan really think such obstacles could never be overcome? Even when doing so might quickly allow a city or nation to dominate the world? His “near-zero prior” seems to come not from any fundamental analysis but, from his strong reliance on intuition; I suspect he would have similarly assigned a very low prior to manned flight in 1850, or to space flight in 1900.

But as I said above, I expect many others agree with his intuition, and I thank him for saying explicitly what others only think.

GD Star Rating
loading...
Tagged as: ,
Trackback URL:
  • http://www.kauffj.com kauffj

    (A relatively minor note, but for those who read your blog via RSS, indentation to indicate a quote does not carry over. However, using the <blockquote> tag or quoting via your editor UI will carry over to RSS readers and has a similar visual effect for those who visit your blog directly.)

  • free_agent

    You write, “But those who think my book too weird mostly stay silent; I’ve heard privately of many who were going to cover the book before they fully realized what it is about.”

    In the end, the marketplace of ideas is a market for attention.

  • AG

    When reviewing a book it is possible to critique -among other things- substance and style. When people have strong emotions about the subject, it becomes easier to distinguish between substantive critiques and style critiques.

    What type of criqtue is “The Age of Em says vanishingly little about the lives of biological humans during the Age of Em” ?

    You’ve addressed them as style critiques, stating “ These are complaints about language and emphasis.” However, some readers can consider it a substantive critique, they are concerned with humans during this time period, and do not feel that you’ve address their desire for information on humans during this period. Equal emphasis seems un-necessary, as Brian points out when bringing up a counterfactual situation where an author patronizes the audience by giving equal treatment to humans. However, for someone interested in Ems and humans but ignorant of the ideas presented in the book, giving a short chapter or a few well headed paragraphs about what humans are doing during this period would be enjoyable and illustrative.

    This book is very honest about its purpose and very severe in its style. If a reader approaches with the hope that they will have their questions answered, they may be disappointed. I found the book easier to read when I re-evaluated my priors and accepted that this books wasn’t really written with the purpose of answering questions readers wanted answered, the author answers questions he wants to answer. For example, I am not one who is “complaining about humans no longer being at the center of the action” but I think with the help of an editor or with some more musing upon the wants of the audience the book could avoid being tedious and not be filled with examples of many other ways people have used the term.

    Currently, trying to put the book into context is still tedious, except the burden of the tedium has been shifted onto the reader. For many that haven’t kept pace with the EM dialogue, this is frustrating…. I imagine many of the critiques you will get orbit around this issue.

    • http://overcomingbias.com RobinHanson

      There IS a short chapter on humans.

      • AG

        Have I overlooked something written beyond CH. 27 Minds, Humans? Ch 27 isn’t the short chapter I’m looking for and I’d wager it isn’t what some of the critics are looking for. This section doesn’t discuss (a)what humans choose to do as retirees or (b) human push back to an em world.

        I’ve been hopping around the book* so I may be missing something, but I haven’t come upon anything about human push back to Ems. This trivializes humans in a way that offends my bias, sure. However, if you have an explanation within the book for why humans will allow this scenario to unfold, I’ve not yet gotten to it. For many readers not answering this question removes the discussion from the realm of possibility.

        I understand that these concerns are beyond the scope of the book. However, for many readers, it is an important subject. Do you need to cater to us? No. Does that leave us with a valid subjective critique? Yes. My intuition brings up images of Eloi and the reclining humans in Wall-e when I think of hyper-productive Ems, not merely “retirees”. Consequently, I love the book, but I would love it more if you would have done some cursory analysis of humans within the world.

        *I appreciate that it is efficiently written and organized in a manner that doesn’t require front to back reading

      • http://overcomingbias.com RobinHanson

        “if you have an explanation within the book for why humans will allow this scenario to unfold, I’ve not yet gotten to it”

        Mostly, no one has been driving the tech train for a long time. It is very hard to coordinate to prevent new tech, and it has only rarely happened.

      • AG

        Thanks. This aligns with my thoughts on tech. Sometimes in conversation I exaggerate the idea for effect stating, “we’re nothing but redundant and mutating bio-memory for tech advancement, humans change and die, but our tech tends to improve/grow in complexity”. Precisely b/c no one is driving the train, it is hard to agree upon where we are going and we can’t assume tech will evolve along a particular path (even if the end point is a singularity). I’m not concerned about humans derailing the train.

        Human push back may create developmental opportunity costs for EMs. I am curious about possible ways humans will try to push back at tech progress. Similar to trees growing around obstacles, I imagine that tech’s ultimate path will be shaped by how human’s react to it.

        For me, this book is missing an element of friction btw humans and Ems as Ems come into existence. I’d like to know either a) why my intuition regarding irrational humans influencing EM opportunity costs and ultimately EM growth is wrong or marginal at best b) why/how any such friction will be smoothed out as time passes.

        Otherwise, this well detailed possibility is just one of many when no one is guiding the tech train.

  • TheBrett

    He does have a point about how the Em Society might become dangerous to regular humans in terms of expropriation, although I think the more likely outcome is that the Em Society becomes really insular and avoids all unnecessary interaction with the broader human society (aside from Ems who are living at human speed and essentially consist of “retirees” in truth).

    • http://juridicalcoherence.blogspot.com/ Stephen Diamond

      Don’t the ems rely on regulars for improved ems? Future humanity may not know how to modify ems besides running them fast, but we they will know a lot more than we already do about selective breeding and eugenics. Would they breed regulars as em templates?

      • http://overcomingbias.com RobinHanson

        There just isn’t enough time. The entire em era would only last a year or two.

  • Lord

    I can understand the weakness of the gdp claim in much the same way people are unimpressed with innovations in cell phones versus flush toilets. We may well count them as large while at the same time being unimpressed with their magnitude. The virtual world could change massively relative to the real world where hard resource constraints will limit growth. The two will operate at two different time scales and growth rates so what occurs within a world will be quite different from what happens between worlds. A fabulous virtual growth rate will mean it will fall in significance relative to the real world even as it grows in size and its real constraints will only bind tighter the more successful it is. .

    • http://lukeparrish.rationalsites.com Luke Parrish

      My understanding is that Robin predicts the amount of physical machinery (in an inclusive sense — computers, MEMS devices, refineries, rockets, etc.) to double itself on a monthly basis or so. (That’s fast enough to build a Dyson sphere in under a decade.) He thinks the reason this doesn’t happen already is mainly lack of human minds to operate such machines. With minds that can be copied at will, that bottleneck disappears.

  • worldvoyageur

    I thought this was relevant to the ‘Age of Em’.


    Half of the employees of Barclays Bank earn less than £25,000 ($40,000) per year. But Barclays also employs 530 “code staff” – people with executive functions – who earn an average of £1.3m each, and there are 1443 who earn more than £500,000 ($800,000). It is likely that “the one per cent” in Barclays Bank earn a total approaching half of the total wage and salary bill of the bank.

    This is from a John Kay speech at the Bank of International Settlements annual conference earlier this month. Presumably, in an em world, many of those high earners would be replaced by ems. The value they deliver would fall back to people in meat space – at least for the short time that there would be people left in meat space.

    The future is already here, just unevenly distributed.

  • brendan_r

    Recently I read Carroll’s new book “The Big Picture” which has a couple chapters on why physicists are so sure that they understand the physics of everyday life, effective field theories, and also what that implies about the possibility of brains being anything but fundamental particles obeying the known rules, i.e we’d notice if they didn’t.

    I understand enough physics to get the gist of this argument but crapped myself when I looked at the mathematical appendix in the back.

    Being that so I accept what people like Carroll and Hanson say the above arguments imply about minds.

    Assuming Bryan’s not equipped to argue the details of fundamental physics either, why does he persist on this philosophy of mind question?

    Sure, everyone’s equipped to dispute someone like Dennett who’s like “I’m not conscious, there is no puzzle.”, which seems deliberately provocative and silly.

    But it seems a breakdown in rationality on Bryan’s part to continue to aggressively dispute technical things like substrate independence, etc.

    Would he actually bet you on these topics – if there were a way to settle them? Or does he play Devil’s Advocate because it’s useful for fleshing things out?

    • http://overcomingbias.com RobinHanson

      I’m pretty sure he’d bet.

    • http://juridicalcoherence.blogspot.com/ Stephen Diamond

      why physicists are so sure that they understand the physics of everyday life, effective field theories, and also what that implies about the possibility of brains being anything but fundamental particles obeying the known rules … it seems a breakdown in rationality on Bryan’s part to continue to aggressively dispute technical things like substrate independence, etc.

      Are you saying that brains being nothing but fundamental physical particles implies that minds can be realized in any substrate? The proviso is that the relevant relationships between fundamental particles has to be duplicated. What’s to say that the substrate is irrelevant to whether the necessary functional relationships are realizable in a different substrate?

      [This may not be relevant to Brian’s objection. He believes in immaterial souls, doesn’t he – even in an afterlife?]

      • Silent Cal

        I’m having trouble seeing what you mean by this. Any Turing-complete substrate can realize the functional relationships in any computable substrate, and the fundamental particles as we know them are computable.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        But are the relevant functional relationship between the fundamental particles necessarily computable?

      • Peter David Jones

        The comoutablity of mind is more of a problem than the omputablity of physics.

  • http://juridicalcoherence.blogspot.com/ Stephen Diamond

    Bryan’s a meathuman chauvinist.

  • http://lukeparrish.rationalsites.com Luke Parrish

    It’s interesting that Bryan Caplan puts so much weight on regulations, zoning, and resource competition as economy-slowing factors. He thinks the introduction of ems would only speed up economic growth by double the current rate (if that).

    I wonder if he would come to a different conclusion if it were described as a predominantly space based industrial economy, like Jeff Bezos has recommended. It appears none of those three factors would have significant effects soon.

  • http://www.truthcoin.info/ psztorc

    “Who thinks torture would make them more productive overall in the long run?”

    There is a big difference between “Historical Torture” and “EM Torture” — the marginal cost may be low, even zero.

    Historically, you had to forge chains and construct whips and pay overseers, etc., and worry about insurrection, moral outrage. However, the EM universe is “just software”, so one you’ve written it once, it probably takes as much resources for the Em-owner to torture EMs as reward them. They can (probably) torture privately and with impunity. Foucalt’s panopticon prison has got nothing on *an entire universe which can literally read your every thought*.

  • David Condon

    Hasn’t Bryan Caplan ever heard of learned helplessness and the limits of coercion?

  • Pingback: X001: Future Express | More on Robin Hanson’s AGE OF EM | Review The Future

  • riona

    i have tried long path tool to fix the problem

  • Pingback: Overcoming Bias : Caplan Audits Age of Em

  • Pingback: Overcoming Bias : Caplan Debate Status