Tag Archives: Ems

Play Will Persist

We live in the third human era, industry, which followed the farming and foraging eras. Each era introduced innovations that we expect will persist into future eras. Yet some are skeptical. They foresee “post-apocalyptic” scenarios wherein civilization collapses, industrial machines are lost, and we revert to using animals like mules and horses for motive power. Where we lose cities and instead spread across the land. We might even lose organized law, and revert to each small band enforcing its own local law.

On the surface, the future scenario I describe in my book The Age of Em looks nothing like a civilization collapse. It has more better bigger tech, machines, cities, and organizations. Yet many worry that in it we would lose an even more ancient innovation: play. As in laughter, music, teasing, banter, stories, sports, hobbies, etc. Because the em era is a more competitive world where wages return to near subsistence levels, many fear the loss of play and related activities. All of life becomes nose-to-the-grindstone work, where souls grind into dust.

Yet the farming and foraging eras were full of play, even though they were also competitive eras with subsistence wages. Moreover, play is quite common among animals, pretty much all of whom have lived in competitive worlds near subsistence levels:

Play is .. found in a wide range of animals, including marsupials, birds, turtles, lizards, fish, and invertebrates. .. [It] is a diverse phenomenon that evolved independently and was even secondarily reduced or lost in many groups of animals. (more)

Here is where we’ve found play in the evolutionary tree:


We know roughly what kind of animals play:

Animals that play often share common traits, including active life styles, moderate to high metabolic rates, generalist ecological needs requiring behavioral flexibility or plasticity, and adequate to abundant food resources. Object play is most often found in species with carnivorous, omnivorous, or scavenging foraging modes. Locomotor play is prominent in species that navigate in three-dimensional (e.g., trees, water) or complex environments and rely on escape to avoid predation. Social play is not easily summarized, but play fighting, chasing, and wrestling are the major types recorded and occur in almost every major group of animals in which play is found. (more)

Not only are humans generalists with an active lifestyle, we have neoteny, which extends youthful features and behaviors, including play, throughout our lives. So humans have always played, a lot. Given this long robust history of play in humans and animals, why would anyone expect play to suddenly disappear with ems?

Part of the problem is that from the inside play feels like an activity without a “useful” purpose:

Playful activities can be characterized as being (1) incompletely functional in the context expressed; (2) voluntary, pleasurable, or self rewarding; (3) different structurally or temporally from related serious behavior systems; (4) expressed repeatedly during at least some part of an animal’s life span; and (5) initiated in relatively benign situations. (more)

While during serious behavior we are usually aware of some important functions our behaviors serve, in play we enter a “magic circle” wherein we feel safe, focus on pleasure, and act out a wider variety of apparently-safe behaviors. We stop play temporarily when something serious needs doing, and also for longer periods when we are very stressed, such as when depressed or starving. These help give us the impression that play is “extra”, serving no other purpose than “fun.”

But of course such a robust animal behavior must serve important functions. Many specific adaptive functions have been proposed, and while there isn’t strong agreement on their relative importance, we are pretty confident that since play has big costs, it must also give big gains:

Juveniles spend an estimated 2 to 15 percent of their daily calorie budget on play, using up calories the young animal could more profitably use for growing. Frisky playing can also be dangerous, making animals conspicuous and inattentive, more vulnerable to predators and more likely to hurt themselves as they romp and cavort. .. Harcourt witnessed 102 seal pups attacked by southern sea lions; 26 of them were killed. ‘‘Of these observed kills,’’ Harcourt reported in the British journal Animal Behaviour, ‘‘22 of the pups were playing in the shallow tidal pools immediately before the attack and appeared to be oblivious to the other animals fleeing nearby.’’ In other words, nearly 85 percent of the pups that were killed had been playing. (more)

Play can help to explore possibilities, both to learn and practice the usual ways of doing things, and also to discover new ways. In addition, play can be used to signal loyalty, develop trust and coordination, and establish relative status. And via play one can indirectly say things one doesn’t like to say directly. All of these functions should continue to be relevant for ems.

Given all this, I can’t see much doubt that ems would play, at least during the early em era, and play nearly as typical humans in history. Sure it is hard to offer much assurance that play will continue into the indefinite future. But this is mainly because it is hard to offer much assurance of anything in the indefinite future, not because we have good specific reasons to expect play to go away.

GD Star Rating
Tagged as: , ,

Social Science Critics

Many critics of Age of Em are critics of social science; they suggest that even though we might be able to use today’s physics or computer science to guess at futures, social science is far less useful.

For example At Crooked Timber Henry Farrell was “a lot more skeptical that social science can help you make predictions”, though he was more skeptical about thinking in terms of markets than in terms of “vast and distributed hierarchies of exploitation”, as these “generate complexities” instead of “ breaking them down.”

At Science Fact & Science Fiction Concatenation, Jonathan Cowie suggests social science only applies to biological creatures:

While Hanson’s treatise is engaging and interesting, I confess that personally I simply do not buy into it. Not only have I read too much SF to think that em life will be as prescriptive as Hanson portrays, but coming from the biological sciences, I am acutely aware of the frailties of the human brain hence mind (on a psychobiological basis). Furthermore, I am uncomfortable in the way that the social science works Hanson draws upon to support his em conclusions: it is an apples and oranges thing, I do not think that they can readily translate from one to the other; from real life sociobiological constructs to, in effect, machine code. There is much we simply do not know about this, as yet, untrodden land glimpsed from afar.

At Ricochet, John Walker suggests we can’t do social science if we don’t know detail stories of specific lives:

The book is simultaneously breathtaking and tedious. The author tries to work out every aspect of em society: the structure of cities, economics, law, social structure, love, trust, governance, religion, customs, and more. Much of this strikes me as highly speculative, especially since we don’t know anything about the actual experience of living as an em or how we will make the transition from our present society to one dominated by ems.

At his blog, Lance Fortnow suggests my social science assumes too much rationality:

I don’t agree with all of Hanson’s conclusions, in particular he expects a certain rationality from ems that we don’t often see in humans, and if ems are just human emulations, they may not want a short life and long retirement. Perhaps this book isn’t about ems and robots at all, but about Hanson’s vision of human-like creatures as true economic beings as he espouses in his blog. Not sure it is a world I’d like to be a part of, but it’s a fascinating world nevertheless.

At Entropy Chat List, Rafal Smigrodzki suggests social science doesn’t apply if ems adjust their brain design:

My second major objection: Your pervasive assumption that em will remain largely static in their overall structure and function. I think this assumption is at least as unlikely as the em-before-AI assumption. Imagine .. you have the detailed knowledge of your own mind, the tools to modify it, and the ability to generate millions of copies to try out various modifications. .. you do analyze this possibility, you consider some options but in the end you still assume ems will be just like us. Of course, if ems are not like us, then a lot of the detailed sociological research produced on humans would not be very applicable to their world and the book would have to be shorter, but then it might be a better one. In one chapter you mention that lesbian women make more money and therefore lesbian ems might make money as well. This comes at the end of many levels of suspension of disbelief, making the sociology/gender/psychology chapters quite exhausting.

At his blog, J Storrs Hall said something similar:

Robin’s scenario precludes some of these concerns by being very specific to a single possibility: that we have the technology to copy off any single particular human brain, we don’t understand them well enough to modify them arbitrarily. Thus they have to operated in a virtual reality that is reasonably close to a simulated physical world. There is a good reason for doing it this way, of course: that’s the only uploading scenario in which all the social science studies and papers and results and so forth can be assumed to still apply.

Most social scientists, and especially most economists, don’t see what they have learned as being quite so fragile. Yes it is nice to check abstract theories against concrete anecdotes, but in fact most who publish papers do little such checking, and their results only suffer modestly from the lack. Yes being non-biological, or messing a bit with brain design, may make some modest differences. But most social science theory just isn’t that sensitive to such details. As I say in the book:

Our economic theories apply reasonably well not only to other classes and regions within rich nations today, but also to other very different nations today and to people and places thousands of years ago. Furthermore, formal economic models apply widely even though quite alien creatures usually populate them, that is, selfish rational strategic agents who never forget or make mistakes. If economic theory built using such agents can apply to us today, it can plausibly apply to future ems.

The human brain is a very large complex legacy system whose designer did not put a priority on making it easy to understand, modify, or redesign. That should greatly limit the rate at which big useful redesign is possible.

GD Star Rating
Tagged as: ,

How Culturally Plastic?

Typical farming behaviors violated forager values. Farmers added marriage, property, war, inequality, and much less art, leisure and travel. 100K years ago if someone had suggested that foragers would be replaced by farmers, critics could easily have doubted that foragers would act like that. But tens of thousands of years was enough time for cultural variation and selection to produce new farming cultures more compatible with the new farming ways.

A typical subsistence farmer from a thousand years ago might have been similarly skeptical about a future industrial world wherein most people (not just elites) pick leaders by voting, have little religion, spend fifteen years of their youth in schools, and are promiscuous, work few hours, abide in skyscrapers, ride in fast trains, cars, & planes, and work in factories and large organizations with much and explicit rules, ranking, and domination. Many of these acts would have scared or offended typical farmers. Even those who knew that tens of millennia was enough to create cultures that embraced farming values might have doubted a few centuries was enough for industry values. But it was.

In my book The Age of Em I describe a world after which it has adapted to brain emulation tech. While I tend to assume that culture has changed to support habits productive in the competitive em world, a common criticism of my book is that the behaviors I posit for the em world conflict with values commonly held today. For example, from Steven Poole’s Guardian review:

Hanson assumes there is no big problem about the continuity of identity among such copies. .. But there is plausibly a show-stopping problem here. If someone announces they will upload my consciousness into a robot and then destroy my existing body, I will take this as a threat of murder. The robot running an exact copy of my consciousness won’t actually be “me”. (Such issues are richly analysed in the philosophical literature stemming from Derek Parfit’s thought experiments about teleportation and the like in the 1980s.) So ems – the first of whom are, by definition, going to have minds identical to those of humans – may very well exhibit the same kind of reaction, in which case a lot of Hanson’s more thrillingly bizarre social developments will not happen. (more)

Peter McCluskey has similar reservations about my saying at least dozens of human children would be scanned to supply an em economy with flexible young minds:

Robin predicts few regulatory obstacles to uploading children, because he expects the world to be dominated by ems. I’m skeptical of that. Ems will be dominant in the sense of having most of the population, but that doesn’t tell us much about em influence on human society – farmers became a large fraction of the world population without meddling much in hunter-gatherer political systems. And it’s unclear whether em political systems would want to alter the relevant regulations – em societies will have much the same conflicting interest groups pushing for and against immigration that human societies have. (more)

Farmers may not have meddled much in internal forager cultures, nor industry in internal farmer culture. But when prior era cultural values have conflicted with key activities of the new era, new eras have consistently won such conflicts. And since the em era should encompass thousands of years of subjective experience for typical ems, there seems plenty of time for em culture to adapt to new conditions. But as humans may only experience a few years during the em era and its preceding transition, it seems more of an open question how far human behaviors would adapt.

We are talking about the em world needing a small number of humans scanned, especially children. Such scans are probably destructive, at least initially. As individual human inclinations vary quite a lot, if the choice is up to individuals, enough humans would volunteer. So the question is if human coordinate enough in each area to prevent this, such as via law. If they coordinate well in most areas, but not in a few other areas, then if there are huge productivity advantages from being able to scan people or kids, the few places that allow it will quickly dominate the rest. And in anticipation of that loss, other places would cave as well. So without global coordination to prevent this, it happens.

Peter talks about the possibility of directly emulating the growth of baby brains all the way from the beginning. And yes if this was easy enough, the em world wouldn’t bother to fight organized human opposition. However, since emulation from conception seems a substantial new capacity, I didn’t feel comfortable assuming it in my book. So I focused on the case where it isn’t possible early on, in which case the above analysis applies.

This whole topic is mostly about: how culturally plastic are we? I’ve been assuming a lot of plasticity, and my critics have been saying less. The academics who most specialize in cultural plasticity, such as anthropologists, tend to say we are quite plastic. So as with my recent post on physicists being confident that there is no extra non-physical feeling stuff, this seems another case where most people have strong intuitions that conflict with expert claims, and they won’t defer to experts.

GD Star Rating
Tagged as: ,

No Short Em Age

The basic premise of my book is that the next big revolution on the scale of the farming and industrial revolutions will come from human level artificial intelligence in the form of brain emulations (ems). Yes, because people have asked I’ve estimated that this will happen within roughly a century, but that estimate isn’t central. The key is that even if ems take many centuries, they will still come before achieving human level artificial intelligence via the usual methods (UAI – via hand-coded algorithms including statistics), and before other social disruptions of this magnitude.

I’ve argued that this premise is plausible because it is hard to imagine social disruptions as big as AI, and because at past rates of progress UAI should take centuries, while ems look like they’ll be ready sooner. Yes, some people are so impressed by recent UAI demos that they think this time is different, so that we will now see an unprecedented burst of progress all the way to full UAI within a decade or two. But I remember past demos that were similarly impressive relative to then-current abilities.

Some people think the basic premise of my book is too weird, while others see it as not weird enough. This post addresses the most common objection I’ve heard from this second group: that even if ems come first, the usual AI will appear a few hours later, making the age of em too short to be worth much consideration.

Now there is certainly one way big ems make full UAI come faster: by speeding up overall economic growth. I’ve suggested the em economy might double every month or faster, and while some doubt this, few who think my book not weird enough are among them.

Since the economy mainly grows today via innovation, our ladder of growth is basically a ladder of overall innovation. We only double the economy when we have on averaged doubled our abilities across all economic sectors. So if the relative rates of economic growth and innovation in different sectors stay the same, then speeding up economic growth means speeding up the rate of progress toward full UAI. (While some expect a larger economy to innovate faster because it has more resources, the steady economic growth rates we’ve seen suggest there are contrary forces, such as picking the low hanging fruit of research first.)

For example, at past rates of UAI progress it should take two to four centuries to reach human level abilities in the typical UAI subfield, and thus even longer in most subfields. Since the world economy now doubles roughly every fifteen years, that comes to twenty doublings in three centuries. If ems show up halfway from now to full human level usual AI, there’d still be ten economic doublings to go, which would then take ten months if the economy doubled monthly. Which is definitely faster UAI progress.

However, ten doublings of the economy can encompass a whole era worthy of study. I’ve argued that ems would typically run fast enough to fit a subjective career of a century or more within an economic doubling time, so that their early career training can remain relevant over a whole career. So ten doublings is at least ten subjective centuries, which is plenty of time for lots of cultural and social change. A whole age of change, in fact.

Some argue that the existence of ems would speed up innovation in general, because ems are smarter and innovation benefits more from smarts than does typical production. But even if true, this doesn’t change the relative rate of innovation in UAI relative to other areas.

Some argue that ems speed up UAI progress in particular, via being able to inspect brain circuits in detail and experiment with variations. But as it can be very hard to learn how to code just from inspecting object spaghetti code from other coders, I’m skeptical that this effect could speed up progress anything like a factor of two, which would be where two (logarithmic) steps on the UAI ladder of progress are now jumped when single steps are on average jumped elsewhere. And even then there’d still be at least five economic doublings in the em era, giving at least five subjective centuries of cultural change.

And we know of substantial contrary effects. First, UAI progress seems driven in part by computer hardware progress, which looks like it will be slower in the coming decades than it has in past decades, relative to other areas of innovation. More important, a big part of em era growth can be due to raw physical growth in production, via making many more ems. If half of em economic growth is due to this process then the em economy makes two (logarithmic) steps of economic growth for every step on the ladder of innovation progress, turning ten ladder steps into twenty doublings. A long age of em.

Some argue that the availability of ems will greatly speed the rate of UAI innovation relative to other rates of innovation. They say things like:

When ems are cheap, you could have a million top (e.g., 100 times average) quality UAI research ems each running at a million times human speed. Since until now we’ve only had a thousand average quality UAI researchers at any one time, UAI progress could be a hundred billion times faster, making what would have taken three centuries now take a tenth of a second. The prize of getting to full UAI first would induce this investment.

There are just so many things wrong with this statement.

First, even if human speed ems are cheap, mega-ems cost at least a million times as much. A million mega-ems are as productive as trillion humans, times whatever factor by which the typical human-speed em is more productive than a typical human. The em economy would have to have grown a whole lot before it is even possible to devote that level of resources to UAI research. So there can be a whole em era before that point.

Second, this same approach seems equally able to speed up progress in any innovation area that isn’t strongly limited by physical process rates. Areas that only moderately depend on physical rates can spend more to compensate, so that their innovation rates slow only modestly. If only a modest fraction of innovation areas were substantially limited by physical rates, that would only speed up UAI progress by a modest factor relative to overall economic growth.

Third, just because some researchers publish many more academic papers than others doesn’t at all mean that young copies of those researchers assigned to other research areas would have published similarly. Ex ante expected researcher quality varies a lot less than ex post observed research publications. Yes, people often vary by larger factors in their ability to do pure math, relative to other abilities, but pure math contributes only a small fraction to overall innovation.

Fourth, it is well known that most innovation doesn’t come from formal research, and that innovations in different areas help each other. Economists have strong general reasons to expect diminishing returns to useful innovation from adding more researchers. Yes, if you double the number of researchers in one area you’ll probably get twice as many research papers in that area, but that is very different from twice as getting much useful progress.

As I mention in my book, in some cases we’ve actually measured how research progress varies with the number of researchers, and it looks more like a square root dependence. In addition, if innovation rates were linear in the number of formal researchers, then given the tiny fraction of such researchers today we’d have to be vastly underinvesting in them, and so nations who invest more in formal research should expect to see much higher rates of economic growth. Yet we don’t actually see much of a relation between economic growth and spending on formal research. (Yes studies vary, so there could be a modest, but not a huge, effect.)

So, in sum, we should expect that useful UAI innovation doesn’t mostly come from formal research, and so doubling the number of UAI researchers, or doubling their speed, doesn’t remotely double useful innovation. We aren’t vastly underinvesting in formal research, and so future parties can’t expect to achieve huge gains by making a huge new investment there. We can expect to see modest gain in UAI innovation, relative to today and to other innovation areas, from an ability to inspect and experiment with ems, and from not being very limited by physical process rates. But these give less than a factor of two, and we should see a factor of two in the other direction from slowing hardware gains and from innovation mattering less for economic growth.

Thus we should expect many doublings of the em era after ems and before human level UAI, resulting in many centuries of subjective cultural change for typical ems. Giving an em era that is long enough to be worth considering. If you want to study whatever comes after the em era, understanding the em era should help.

GD Star Rating
Tagged as: , ,

My Caplan Turing Test

At lunch today Bryan Caplan and I dug a bit into our disagreement, and now I’ll try to summarize his point of view. He can of course correct me.

Bryan sees sympathy feelings as huge influences on social outcomes. Not just feelings between people who know each other well, but also distant feelings between people who have never met. For example, if not for feelings of sympathy:

  1. Law and courts would often favor different disputants.
  2. Free workers would more often face harsh evaluations, punishments, and firing.
  3. Firm owners and managers would know much better which workers were doing good jobs.
  4. The US would invade and enslave Canada tomorrow.
  5. At the end of most wars, the victors would enslave the losers.
  6. Modern slaves would earn their owners much more than they would have as free workers.
  7. In the past, domestic, artisan, and city slaves, who were treated better than field slaves, would have been treated much more harshly.
  8. The slave population would have fallen less via gifts or purchase of freedom.
  9. Thus most of the world population today would be slaves.

These views are, to me, surprisingly different from the impression I get from reading related economics literatures. Bryan says I may be reading the wrong ones, but he hasn’t yet pointed me to the correct ones. As I read them, these usual economics literatures give different impressions:

  • Law and economics literature suggests efficiency usual decides who wins, with sympathy distortions having a real but minor influence.
  • Organization theory literature suggests far more difficulties in motivating workers and measuring their performance.
  • Slavery literature suggests slaves doing complex jobs were treated less harshly for incentive reasons, and would not have earned much more if treated more harshly. Thus modern slaves would also not earn much more as slaves.

Of course even if Bryan were right about all these claims, he needn’t be right in his confident opinion that the vast majority of biological humans will have about as much sympathy for ems as they do for mammals, and thus treat ems as harshly as we treat most mammals.

This sympathy-driven view doesn’t by itself predict Caplan’s strong (and not much explained) view that ems would also be very robot-like. But perhaps we might add to it a passion for domination – people driven by feelings to treat nicely creatures they respect might also be driven by feelings to dominate creatures they do not respect. Such a passion for dominance might induce biological humans to force ems to into ultra docility, even if that came at a productivity cost.

Added 28July2016: Caplan grades my summary of his position. I’m mostly in the ballpark, but he elaborates a bit on why he thinks em slaves would be docile:

Docile slaves are more profitable than slaves with attitude, because owners don’t have to use resources to torture and scare them into compliance.  That’s why owners sent rebellious slaves to “breakers”: to transform rebellious slaves into docile slaves.  Sci-fi is full of stories about humans genetically engineered to be model slaves.  Whole brain emulation is a quicker route to a the same destination.  What’s the puzzle?

For docility to be such a huge priority, relative to other worker features, em rebellion must happen often and impose big frequent costs. Docility doesn’t seem to describe our most productive workers today well, nor does it seem well suited when you want workers to be creative, think carefully, take the initiative, or persuade and inspire others. Either way, either frequent costly rebellions or extreme docility, create big disadvantages of slaves relative to free workers, and so argues against most ems being slaves.

GD Star Rating
Tagged as: , ,

Oarsman Pay Parable

Imagine an ancient oarsman, rowing in a galley boat. Rowing takes effort, and risks personal injury, so all else equal an oarsman would rather not row, or row only weakly. How can his boss induce effort?

One simple approach is to offer a very direct and immediate incentive. Use slaves as rowers, and have a boss watch them, whipping any who aren’t rowing as hard as sustainably possible. This actually didn’t happen much in the ancient world; galley slaves weren’t common until the 1500s. But the idea is simple. And of course the same system could also work with cash; usually make positive payments for work, but sometimes fine those you discover aren’t working hard enough. Of course the boss can’t watch everyone all the time. But with a big enough penalty when caught, it might work.

Now imagine that the boss can’t watch each individual oarsman, but can only see the overall speed of the ship. Now the entire crew must be punished together, all or none of them. The boss might try to improve the situation by empowering oarsmen to punish each other for not rowing hard enough, and that might help, but rowers would also use that power for other ends, creating costs.

An even worse case is where the boss can only see how long it takes for the boat to reach its destination. Here the boss might reward the crew for a short trip, and punish them for a long one, but a great many other random factors will influence the length of the trip. Why bother to work hard, if it makes little difference to your chance of reward or punishment?

There is a general principle here. As we add more noise to the measurement of relevant outcomes visible to the ultimate boss, the harder it is to use incentives tied to such outcomes to incentivize rowers. This is true regardless of the type of incentives used. Yes, the lower the worst outcome, and the higher the best outcome, that the boss can impose, the stronger incentives can be. But even the strongest possible incentives can fail when noise is high.

Yes, one can create layers of bosses, with the lowest bosses able to see specifics best. But it can be hard to give lower bosses good incentives, if higher bosses can’t see well.

Another problem is if the boss doesn’t know just how hard each oarsman is capable of rowing. In this case most oarsmen get some slack, so that they aren’t punished for not doing more than they can. This is just one example of an “information rent”. In general, such rents come from any work-relevant info that the worker has that the boss can’t see. If rowers need to synchronize their actions with each other or with waves or wind or time of day. If a ship captain needs to choose the ship’s route based info on weather and pirates. If a captain needs to treat different cargo differently in different conditions. If a captain need to make judgements about whether to wait longer in port for more cargo.

In general, when you want a worker to see some local condition, and then take an action that depends on that condition, you must pay some extra rent. So the more relevant info that workers get, the more choices they make, and the more that rides on those choices, the more workers gain in info rents.

A related issue is the scope for sabotage. Angry resentful workers can seek hidden ways to hurt their bosses and ventures. So the more hard-to-detect ways workers have to hurt things, the more bosses want to treat them well enough to avoid anger and resentment. Pained, sullen, or depressed workers can also hurt the mood of co-workers, suppliers, customers, and investors whom they contact. And the threat of pain can stress workers, making it harder for them to think clearly and well. These issues tend to argue against often using beatings and pain for motivation, even if such things allow stronger incentives by expanding the range of possible outcomes.

Overall, these issues are bigger for more “complex” work, i.e., for more cognitive work, work that adapts more to diverse and new local conditions, and work in larger organizations. In the modern world, jobs have been getting more complex in these ways, and the organization and work literature I’ve read suggests that finding good work incentives is a central problem in modern organizations, and that more complex work is a big reason why modern workplaces substitute broad incentives and good treatment for the detailed and harsh rules and monitoring more common in past eras.

The literature I’ve read on the economics of slavery also uses job complexity to explain the severity of treatment of slaves. Slaves in artisan jobs, in cities, and in households were treated better than field slaves, arguably because of job complexity. They were beaten less, and paid more, and might eventually buy their own freedom.

Bryan Caplan has argued that ems would be treated harshly as slaves: Continue reading "Oarsman Pay Parable" »

GD Star Rating
Tagged as: , ,

Caplan Debate Status

In this post I summarize my recent disagreement with Bryan Caplan. In the next post, I’ll dive into details of what I see as the key issue.

I recently said:

If you imagine religions, governments, and criminals not getting too far out of control, and a basically capitalist world, then your main future fears are probably going to be about for-profit firms, especially regarding how they treat workers. You’ll fear firms enslaving workers, or drugging them into submission, or just tricking them with ideology.

Because of this, I’m not so surprised by the deep terror many non-economists hold of future competition. For example, Scott Alexander (see also his review):

I agree with Robin Hanson. This is the dream time .. where we are unusually safe from multipolar traps, and as such weird things like art and science and philosophy and love can flourish. As technological advance increases, .. new opportunities to throw values under the bus for increased competitiveness will arise. .. Capitalism and democracy, previously our protectors, will figure out ways to route around their inconvenient dependence on human values. And our coordination power will not be nearly up to the task, assuming something much more powerful than all of us combined doesn’t show up and crush our combined efforts with a wave of its paw.

But I was honestly surprised to see my libertarian economist colleague Bryan Caplan also holding a similarly dark view of competition. As you may recall, Caplan had many complaints about my language and emphasis in my book, but in terms of the key evaluation criteria that I care about, namely how well I applied standard academic consensus to my scenario assumptions, he had three main points.

First, he called my estimate of an em economic growth doubling time of one month my “single craziest claim.” He seems to agree that standard economic growth models can predict far faster growth when substitutes for human labor can be made in factories, and that we have twice before seen economic growth rates jump by more than a factor of fifty in a less than previous doubling time. Even so, he can’t see economic growth rates even doubling, because of “bottlenecks”:

Politically, something as simple as zoning could do the trick. .. the most favorable political environments on earth still have plenty of regulatory hurdles .. we should expect bottlenecks for key natural resources, location, and so on. .. Personally, I’d be amazed if an em economy doubled the global economy’s annual growth rate.

His other two points are that competition would lead to ems being very docile slaves. I responded that slavery has been rare in history, and that docility and slavery aren’t especially productive today. But he called the example of Soviet nuclear scientists “powerful” even though “Soviet and Nazi slaves’ productivity was normally low.” He rejected the relevance of our large literatures on productivity correlates and how to motive workers, as little of that explicitly includes slaves. He concluded:

If, as I’ve argued, we would copy the most robot-like people and treat them as slaves, at least 90% of Robin’s details are wrong.

As I didn’t think the docility of ems mattered that much for most of my book, I challenged him to audit five random pages. He reported “Robin’s only 80% wrong”, though I count only 63% from his particulars, and half of those come from his seeing ems as very literally “robot-like”. For example, he says ems are not disturbed by “life events”, only by disappointing their masters. They only group, identify, and organize as commanded, not as they prefer or choose. They have no personality “in a human sense.” They never disagree with each other, and never need to make excuses for anything.

Caplan offered no citations with specific support for these claims, instead pointing me to the literature on the economics of slavery. So I took the time to read up on that and posted a 1600 summary, concluding:

I still can’t find a rationale for Bryan Caplan’s claim that all ems would be fully slaves. .. even less .. that they would be so docile and “robot-like” as to not even have human-like personalities.

Yesterday, he briefly “clarified” his reasoning. He says ems would start out as slaves since few humans see them as having moral value:

1. Most human beings wouldn’t see ems as “human,” so neither would their legal systems. .. 2. At the dawn of the Age of Em, humans will initially control (a) which brains they copy, and (b) the circumstances into which these copies emerge. In the absence of moral or legal barriers, pure self-interest will guide creators’ choices – and slavery will be an available option.

Now I’ve repeatedly pointed out that the first scans would be destructive, so either the first scanned humans see ems as “human” and expect to not be treated badly, or they are killed against their will. But I want to focus instead on the core issue: like Scott Alexander and many others, Caplan sees a robust tendency of future competition to devolve into hell, held at bay only by contingent circumstances such as strong moral feelings. Today the very limited supply of substitutes for human workers keeps wages high, but if that supply were to greatly increase then Caplan expects that without strong moral resistance capitalist competition eventually turns everyone into docile inhuman slaves, because that arrangment robustly wins productivity competitions.

In my next post I’ll address that productivity issue.

GD Star Rating
Tagged as: , ,

The Future of Slavery

Bryan Caplan made strong, and to me incredible, claims that econ consensus predicts all ems would be fully slaves with no human personality. As he won’t explain his reasoning, but just says to read the slavery literature, I’ve done a quick lit review, which I now summarize, and then apply quickly to the future in general, and to ems in particular.

The ability to control your pain, actions, and income are distinct property rights. When someone else owns them all, you are said to be a slave, especially if they allocate these rights via something close to a “full control” package. In this package, you have little control over assets or actions. Pain is usually threatened, and often implemented, to force specific disliked but demanded actions. (Pain was used more on children than adults.) Think of rowing for a galley ship, digging up silver in a mine, picking cotton, or advancing on a simple war front line.

A second “mixed control” package allocates these rights by letting you retain control over many action details, only rarely causing pain, and letting you earn a residual income or status. This scenario was more common for domestic slaves, for slaves with better options for sabotage or escape, and for complex jobs where motivation matters more, via worker discretion, responsibility, attentiveness, pleasantness, intelligence, or creativity. By collecting a residual income, slaves might eventually buy their freedom. Free people have often sold this package of rights for short durations in traditional jobs. The main difference is your ease of changing jobs; the harder it is to change jobs, the more like this kind of slave you are.

In a third “debt” package, you must pay off a loan but are otherwise mostly free to choose your own job, location, and living arrangements. The option to impose pain is reserved for rare situations. Closely related is “share cropping” wherein the owner demands a percentage of income earned. Some combination of a fixed payment plus a percentage of income was a common scenario for slaves in southern US cities. This is also the usual way state rulers extort the locals they “own” via taxation. Many people voluntarily choose to go into debt, and sell percentages of their business income, and most legal systems reserve the right to impose pain in rare situations, a situation most people are okay with.

A fourth “ransom” approach sells these rights back to some combination of you and your associates. Often this converts these rights into debt held by someone who is better able to motivate and monitor you.

Many considerations influence the efficiency of these allocations, including costs of monitoring and restraint, losses from theft, rebellion, escape, and sabotage, individual preferences for pain, status, autonomy, and work style, effects of pain, status, and control on motivation and focus, information rents from workers being better aware of work details, complementary investments in training and capital, who knows better and has better incentives to use control rights, and signaling status, productivity, etc. to outsiders.

Historically, even when slaves were common, they were usually a minority of the population. (Beware, the term “slave” is used in different ways.) About 10% in the Roman Empire and US south. Foragers didn’t do slaves at all. About 0.3% of the world is in slavery today, mostly in forms of debt bondage.

The common existence of slavery that wasn’t converted immediately into debt or ransom does suggest that it was sometimes locally efficient as a resource allocation, ignoring larger social externalities, even given substantial costs of monitoring, enforcement, and worse motivation and allocation of skills.

Sometimes during hard times people would sell themselves or their children into slavery; better to be fed than dead. Sometimes slaves were created as collateral for loans, and freed when the loan was paid. Sometimes slavery was the contractual result of a failure to pay loans. Sometimes people sold themselves into slavery for a limited time, as with apprenticeships and indentured servitude.

But historically, slaves were mostly created in war. Drafted soldiers are slave-like. When a winning side didn’t expect to hold the territory, and feared leaving the vanquished to recover then retaliate, their remaining options were death or slavery. But slaves were only valuable when delivered to a useable location. So the worse treatment of slaves has been in transit immediately after capture.

Slave populations usually dwindled until replenished by war, probably because through most of history interest rates were too high to justify the long term investment of raising human children. Domesticated crops and animals grow much quicker. This same short term focus also often induced slave owners to work their slaves to death. A short term focus was often increased by distant ownership, as local manager’ incentives were tied more to immediate production. Workings slaves to death induced more slave revolts.

The US south was unusual in that it grew long-lived slaves from birth. Interest rates were unusually low, peace lasted long, and once US law forbad importing slaves, owners were highly motivated to preserve their big plantation industry. Slaves weren’t converted into debt perhaps because of credit market failures, or more plausibly because the full control approach was especially productive on plantations. (The sex story is overrated, as only 1-2% of slave babies were fathered by white men.)

That is, on plantations slaves plausibly produced more when threatened with pain, even if their utility was lower. The fact that humans can feel strongly disliked pain while living a long productive life and successfully reproducing does suggest that our pain signals are biologically maladaptive. But given how different is the modern world from the one where our pain signals evolved, we should expect this sort of thing sometimes.

Data on US south slave prices tells us what was valued in slaves then. For adults, age was bad, as were slaves from distant places within the US, and slaves that the owner chose to sell, as opposed to being forced to sell. New slaves imported from overseas were no more or less valued. It was good to be male, light-skinned, have artisan skills, and be guaranteed not to be sick or run away.

I didn’t find any data on slaves and docility, though I did find how docility fits into the standard five factor personality framework. Docility is lumped with “submissive, dependent, pliant” as part of “passivity”, which correlates most strongly and positively with neuroticism, but also positively with agreeableness and negatively with openness. In general only the agreeable part suggests more productivity in most jobs today; neurotic people are less productive, and the effect of openness depends more on job type.

What is there to dislike about slavery? The war and theft that cause slavery are clearly lamentable. And the possibility of slavery increases the range of possible inequality, at least if you ignore the dead. But the full control allocation package seems the main reason to dislike slavery. Other packages seem much closer to those resulting from free choices, and when they result from free choices they don’t seem strongly objectionable.

Today slavery, especially full control slavery, is discouraged not only via moral censure and political coordination, but also by stronger nation-states, few wars, better credit markets, increasing wealth, increasing vulnerability to sabotage, more automation, and more complex jobs. The only contrary factors I can think of are easier monitoring and preventing escape. If all these trends continue in the same relative proportions, we should expect a continued decline in slavery.

In the world of my book, The Age of Em, many of these trends continue. Nation-states and credit markets get stronger, and war remains rare. Automation advances, and jobs get even more complex, with motivation and sabotage mattering even more. Monitoring and preventing escape also get easier.

Individual em incomes do fall, which gives a thicker lower tail of outcomes, and in traditional societies that allowed slavery this low tail was often filled with slaves. However, ems can fall via running slower while remaining free, and this option would reduce the fraction that fall into slavery, even if slavery were allowed.

Ems are initially created via destructive scanning of high income human volunteers at the peak of their careers, in a world that forbids slavery. Soon after they are destructive scans of the most promising young children. So these volunteers do not expect to become slaves, and the world around them, being like ours, initially tries to discourage that transition.

However, since a lot changes we can’t offer much assurance that attitudes toward slavery don’t change. Also, labor supply factors matter a lot less; if even one productive em is enslaved, and slavery is allowed, then copies of it could fill a whole slave sector. What matters far more is demand, i.e., what are the more efficient ways to allocate labor? If allowed, there are probably some jobs where full control slavery is more efficient; the em world is big, with many corners. But most jobs are complex, where the full control scenario is inefficient. And the debt or mixed control allocations that are more efficient for typical jobs are probably not substantially more efficient under slavery, as slavery hurts motivation. Debt should be good enough.

So, bottom line, after a quick review of the econ of slavery literature, I still can’t find a rationale for Bryan Caplan’s claim that all ems would be fully slaves. Ancient society never got close to that state of affairs. And I see even less rationale for his claim that they would be so docile and “robot-like” as to not even have human-like personalities. Which is his main reason for saying 80% of my book is wrong. Neither the literatures on choosing employees today, nor that on choosing slaves in the past, put much emphasis on docility. And even if they did, the idea that they’d emphasize it so much as to eliminate human personality, that just sounds crazy.

So Bryan, how about actually giving an argument, instead of waving your hands in the general direction of the literature?

GD Star Rating
Tagged as: , ,

Henry Farrell on Age of Em

There is a difference between predicting the weather, and predicting climate. If you know many details on current air pressures, wind speeds, etc, you can predict the weather nearby a few days forward, but after weeks to months at most you basically only know an overall distribution. However, if there is some fundamental change in the environment, such as via carbon emissions, you might predict how that distribution will change as a result far into the future; that is predicting climate.

Henry Farrell, at Crooked Timber, seems to disagree with Age of Em because he thinks we can only predict social weather, not social climate:

Tyler Cowen says .. Age of Em .. won’t happen. I agree. I enjoyed the book. .. First – the book makes a strong claim for the value of social science in extrapolating likely futures. I am a lot more skeptical that social science can help you make predictions. .. Hanson’s arguments seem to me to rely on a specific combination of (a) an application of evolutionary theory to social development with (b) the notion that evolutionary solutions will rapidly converge on globally efficient outcomes. This is a common set of assumptions among economists with evolutionary predilections, but it seems to me to be implausible. In actually existing markets, we see some limited convergence in the short term on e.g. forms of organization, but this is plausibly driven at least as much by homophily and politics as by the actual identification of efficient solutions. Evolutionary forces may indeed lead to the discovery of new equilibria, but haltingly, and in unexpected ways. .. This suggests an approach to social science which doesn’t aim at specific predictions a la Hanson, so much as at identifying the underlying forces which interact (often in unpredictable ways) to shape and constrain the range of possible futures. ..

In the end, much science fiction is doing the same kind of thing as Hanson ends up doing – trying in a reasonably systematic way to think through the social, economic and political consequences of certain trends, should they develop in particular ways. The aims of extrapolationistas and science fiction writers aims may be different – prediction versus constrained fiction writing but their end result – enriching our sense of the range of possible futures that might be out there – are pretty close to each other. .. it is the reason I got value from his book. ..

So Hanson’s extrapolated future seems to me to reflect an economist’s perspective in which markets have priority, and hierarchy is either subordinated to the market or pushed aside altogether. The work of Hannu Rajaniemi provides a rich, detailed, alternative account of the future in which something like the opposite is true .. [with] vast and distributed hierarchies of exploitation. .. Rajaniemi’s books .. provide a rich counter-extrapolation of what a profoundly different society might look like. .. I don’t know what the future will look like, but I suspect it will be weird in ways that echo Rajaniemi’s way of thinking (which generates complexities) rather than Hanson’s (which breaks them down).

If we can only see forces that shape and constrain the future, but not the distribution of future outcomes, what is the point of looking at samples from the “range of possibilities”? That only seems useful if in fact you can learn things about that range. In which case you are learning about the overall distribution. Isn’t Farrell’s claim about more future “hierarchies of exploitation” relative to “markets” just the sort of overall outcome he claims we can’t know? (Rajaniemi blurbed and likes my book, so I don’t think he sees it as such a polar opposite. And how does hierarchy “generate complexities” while markets “break them down”?) Is Farrell really claiming that there is no overall tendency toward more efficient practices and institutions, making moves away from them just as likely as moves toward them? Are all the insights economic historians think they have gained using efficiency to understand history illusory?

My more charitable interpretation is that Farrell sees me as making forecasts much more confidently than I intend. While I’ve constructed a point prediction, my uncertainty is widely distributed around that point, while Farrell sees me as claiming more concentration. I’ll bet Farrell does in fact see a tendency toward efficiency, and he thinks looking at cases does teach us about distributions. And he probably even thinks supply and demand is often a reasonable first cut approximation. So I’m guessing that, with the right caveat about confidence, he actually thinks my point prediction makes a useful contribution to our understanding of the future.

One clarification. Farrell writes:

One of the unresolved tensions .. Are [ems] free agents, or are they slaves? I don’t think that Hanson’s answer is entirely consistent (or at least I wasn’t able to follow the thread of the consistent argument if it was). Sometimes he seems to suggest that they will have successful means of figuring out if they have been enslaved, and refusing to cooperate, hence leading to a likely convergence on free-ish market relations. Other times, he seems to suggest that it doesn’t make much difference to his broad predictive argument whether they are or are not slaves.

Much of the book doesn’t depend on if ems are slaves, but some parts do, such as the part on how ems might try to detect if they’ve been unwittingly enslaved.

GD Star Rating
Tagged as: ,

Unauthorized Topics

Tyler posted:

Do I think Robin Hanson’s “Age of Em” actually will happen? A reader has been asking me this question, and my answer is…no! Don’t get me wrong, I still think it is a stimulating and wonderful book. .. But it is best not read as a predictive text, much as Robin might disagree with that assessment. Why not? I have three main reasons, all of which are a sort of punting, nonetheless on topics outside one’s areas of expertise deference is very often the correct response. Here goes: 1. I know a few people who have expertise in neuroscience, and they have never mentioned to me that things might turn out this way.

I titled my response Tyler Says Never Ems, but on twitter he objected:

“no reason to think it will happen” is best summary of my view, not “never will happen.”
…that was one polite way of saying I do not think the scientific consensus is with you on this issue…

I responded:

How does that translate into a probability?
You have to clarify the exact claim you have in mind before we can discuss what the scientific consensus says about it.

But all he would answer is:


Now at GMU econ we often have academics who visit for lunch and take the common academic stance of reluctance to state opinions which they can’t back up with academic evidence. Tyler is usually impatient with that, and pushes such visitors to make best estimates. Yet here it is Tyler who shows reluctance. I hypothesize that he is following this common principle:

One does not express serious opinions on topics not yet authorized by the proper prestigious people.

Once a topic has been authorized, then unless a topic has a moral coloring it is usually okay to express a wide range of opinions on it; it is even often expected that clever people will often take contrarian or complex positions, sometimes outside their areas of expertise. But unless the right serious people have authorized a topic, that topic remains “silly”, and can only be discussed in a silly mode.

Now sometimes a topic remains unauthorized because serious people think everything about it has a low probability. But there are many other causes for topics to be seen as silly. For example, sex was long seen as a topic serious people didn’t discuss, even though we were quite sure sex exists. And even though most everyone is pretty sure aliens must exist out there somewhere, aliens remain a relatively silly subject.

In the case of ems, I interpret Tyler above as noting that the people who seem to him the proper authorities have not yet authorized serious discussion of ems. That is what he means by pointing to experts, saying “no reason” and “scientific consensus,” and yet being unwilling to state a probability, or even clarify which claim he rejects, even though I argued a 1% chance is enough. It explains his initial emphasis on treating my book metaphorically. This is less about probabilities, and more about topic authorization.

Compare the topic of ems to the topic of super-intelligence, wherein a single hand-coded AI quickly improves itself so fast that it can take over the world. As this topic seems recently endorsed by Elon Musk, Bill Gates, and Steven Hawking, it is now seen more as an authorized topic. Even though, if you are inclined to be skeptical, we have far more reasons to doubt we will eventually know how to hand-code software as broadly smart as humans, or vastly better than the entire rest of the world put together at improving itself. Our reason for thinking ems eventually feasible is far more solid.

Yet I predict Tyler would more easily accept an invitation to write or speak on super-intelligence, compared to ems. And I conclude many readers see my book primarily as a bid to put ems on the list of serious topics, and they doubt enough proper prestigious people will endorse that bid. And yes, while if we could talk probabilities I think I have a pretty good case, even my list of prestigious book blurters probably aren’t enough. Until someone of the rank of Musk, Gates, or Hawking endorses it, my topic remains silly.

GD Star Rating
Tagged as: , ,