As many have noted, ours in an era of ideological polarization. On topics where there are strong emotions, we tend to gravitate to extremes, and are less interest in intermediate positions. Which is a problem for my book; while most see it as too weird, others mostly see it as not weird enough. A tech futurism minority expects to soon see very rapid progress in artificial intelligence and machine learning, and so see brain emulations as too slow and inefficient compared to the super-intelligence they foresee. And the majority to whom that seems pretty crazy also seem brain emulations as similarly crazy; they don’t care much if ems seem a bit less crazy.
At least future tech enthusiasts who think my book not weird enough are willing to write reviews to say so. But those who think my book too weird mostly stay silent; I’ve heard privately of many who were going to cover the book before they fully realized what it is about. So I thank my college Bryan Caplan for being willing to say what others won’t, in his critical review. His review is long, with ten criticisms. This response will also be long, going point by point.
Six of his ten objections seem to be mainly about my language. (His review is indented, and often contains book quotes; my replies are not.)
1. Robin only pretends to dodge philosophy of mind. .. He tacitly accepts an extreme version of “Ems are just as human as you or me” – and builds the whole book on this assumption. The tell-tale sign: The Age of Em says vanishingly little about the lives of biological humans during the Age of Em! .. he seems so wedded to this philosophical (not social scientific) position that he can’t even feign agnosticism. What would feigning look like? Split the book evenly between discussion of the lives of biological humans during the Age of Em and ems during the Age of Em.
These are complaints about language and emphasis. On language, since I’m constantly applying human and social sciences to ems it would have been quite awkward to use any other than our usual language for describing people. It is hard to imagine a readable book where words like “people” are constantly replaced with phrases like “machines that act like humans but do remember I’m not making any philosophical claims here.” On emphasis, very little happens to biological humans during the em era, so equal emphasis would mean a very short book. That conflicts with my goal of showing just how much I can say about this scenario.
3. Robin has a bizarre definition of “marginalized” .. biological humans .. They’ll be outnumbered, and perform little “hands-on” work. But they’ll be fabulously wealthy and ultimately in charge.” .. 5. Robin’s conclusions only sound “taboo” because he’s using language strangely. .. Robots will “dominate” us no more than rank-and-file workers “dominate” shareholders.
As I mentioned in a previous post, many have reacted to talks I’ve given by complaining about humans no longer being at the center of action, even when they understand that biological humans could for a while own most of the em world, and thus direct how spare resources are spent. I used words like those people use to acknowledge their concerns.
4. Contrary to Robin’s suggestion, there’s near-zero correlation between income and conservatism. .. “subsistence farmers tend to have values more like those of poor/conservative people today.” .. Robin could say he’s defining “conservatism” in a technical or apolitical way. But when you’re writing for an audience, the author rightly bears the burden of highlighting non-standard usage.
In that context I had just cited studies on strong correlations between the culture and wealth of nations, and I had just explained in quite some detail the kind of “conservatism” I meant there. It is true that I didn’t mention explicitly there that the word “conservatism” is used in many different ways. But the book would be a lot more tedious if every time I introduced and used a term I explained the many other ways people have used that term.
7. Robin’s efforts to calm readers’ fear of the future consistently backfire. Example:
Readers of this book may find near subsistence wages to be a strange and perhaps scary prospect. So it is worth remembering that such wages in effect applied to almost all animals who ever lived, to almost all humans before a few hundred years ago, and for a billion humans still today. Historically, it is by far the usual case.
Imagine a middle-class American’s child is destined to earn a subsistence wage. Would it make the parent feel better to hear, “No big deal, your child will face the same fate as almost every animal who ever lived, almost all humans before a few hundred years ago, and a billion humans today”? No, even worse!
Saying something is “worth remembering” just means that it may change how you think about that thing; it is not the same as saying “don’t worry.” It isn’t my job to make readers like the age of em, but it is my job to make sure they keep important considerations in mind.
10. Robin’s argument against the Terminator scenario is much weaker than it looks. His words:
A reasonable hope is that ordinary humans become the retirees of this new world. .. ems may be reluctant to expropriate or exterminate ordinary humans if ems rely on the same or closely interconnected legal, financial, and political systems as humans, and if ems retain many direct social ties to ordinary humans.
The problem: As Robin explains, in one human year, ems experience millennia. So even if each generation of ems only has a .5% chance of expropriating humanity, the chance of expropriation per human year is around 40%.
Bryan misreads me as trying to offer more reassurance than I can. I was clear that even if humans survive the year or two that comprises the age of em, I can say little about what might happen after that. “A reasonable hope” is quite different from “don’t worry.” I would be remiss if I didn’t at least point readers in the direction of a reasonable hope, even if I can offer few guarantees.
Two of Bryan’s objections seem to make opposite points on human constancy:
2. Robin exaggerates how dramatically humans have changed over time: .. `Historical fiction misleads you, showing your ancestors as more modern than they were.’ .. And contrary to Robin, I see a largely constant human nature. Characters in Shakespeare seem as credible to me.
This is a tiny point in the introduction, and little else in the book depends on it. Bryan has read enormous amounts of history, making him an exception mislead far less by historical fiction than are most people. My warning is to them, not him.
6. Robin greatly overstates the difference between his “em scenario” and the “generalized AI scenario.” How so? The Age of Em makes numerous arguments by analogy: Since humans typically do X in situation Y, and ems are copies of humans, ems will also typically do X in situation Y. But he also keeps telling us that only a tiny hand-picked sub-sample of humans will be copied. The obvious question: Why wouldn’t ems largely be copies of the most “robot-like” humans – humble workaholics with minimal personal life, content to selflessly and uncomplainingly serve their employers? This in turn implies that most of Robin’s “detail” is roughly the opposite of what would really happen.”
We have already seen a lot of historical variation in both how much workers submit at work, and in how many hours they work. As I discuss in the book, from around 1820 to 1850 in the U.S., France, and Germany, men worked at jobs an average of 68 to 75 hours per week. And workers in rich nations today accept far more explicit dominance and ranking at work than most of foragers and farmers would have accepted. Even so, there is plenty of commonality between these past and current humans, enough to justify our usual practice of using analogy with humans today to understand past humans. Similarly, even if ems are somewhat more submissive and workaholic than workers today, there should still be enough commonality for analogies to be useful in understanding ems. And as I explained in my response to Scott Alexander, extreme submission and workaholic scenarios seem implausible.
Bryan’s last two objections, on economics, are the ones I take most seriously.
8. Robin greatly overstates the quality of life for ems. .. Why wouldn’t ems’ creators use the threat of `physical hunger, exhaustion, pain, sickness, grime, hard labor, or sudden unexpected death’ to motivate the ems? Robin elsewhere talks about `torturing’ ems, so why not?” .. Modern systems of slave labor – see Stalinist Russia and Nazi Germany – used pain freely, because the penalty for quitting was death.
I was careful not to claim that ems would not be slaves. I just suggested that we didn’t have good reasons to expect most being slaves. Most people in history haven’t been slaves, even when competition has been strong. There are still some slaves in the world today, and they aren’t known for being spectacularly productive workers due to frequent use of torture and pain. Nor were Stalinist Russia and Nazi Germany slaves known for stellar productivity. We have a larger literature on motivating workers, and threats of pain only seem to be useful for the most routine and physical of labor. When you seek workers to be creative, think carefully, take the initiative, or persuade and inspire others, you mostly seek other motivations. Our best analogy for ems should be the few hundred most productive people in the world today, and threats of pain are not remotely what motivate them. Who thinks torture would make them more productive overall in the long run?
9. Robin’s arguments for his single craziest claim – global GDP will double every “month, week, day, or even faster” – are astoundingly weak. .. In the real world, however, there are literally hundreds of bottlenecks that radically retard this kind of growth. Politically, something as simple as zoning could do the trick. .. the most favorable political environments on earth still have plenty of regulatory hurdles .. we should expect bottlenecks for key natural resources, location, and so on. .. This alleged “concrete clue” is nothing compared to the bona fide “concrete clue” that almost all fantastic claims are false. And the idea that the global economy will start doubling on a monthly basis is fantastically fantastic. This has to be the least Bayesian part of the book: We start with a claim with a near-zero prior probability, make a couple of flimsy arguments, and somehow end up with a high posterior probability.
One could have similarly argued that fundamental growth bottlenecks must prevent the previous observed huge jumps in growth rates, such as from foraging to farming, or farming to industry. And plausibly related obstacles did prevent those eras from starting as soon as they might have. But eventually obstacles were overcome. No doubt our current economy tolerates many delays that would have to be cut to enable much faster growth, and the em economy won’t start as early as it might because of regulatory and other delays. My book is mainly about what happens once those obstacles are overcome. Does Bryan really think such obstacles could never be overcome? Even when doing so might quickly allow a city or nation to dominate the world? His “near-zero prior” seems to come not from any fundamental analysis but, from his strong reliance on intuition; I suspect he would have similarly assigned a very low prior to manned flight in 1850, or to space flight in 1900.
But as I said above, I expect many others agree with his intuition, and I thank him for saying explicitly what others only think.