Author Archives: Robin Hanson

Alexander on Age of Em

If I ever have an executioner, I want him to be Scott Alexander. Alexander has such a winning way with words that I and his many fans enjoy him even when we disagree. I’d hardly notice my destination as his pleasing patter entranced me while we took the long way around to the gallows.

So I am honored that Alexander wrote a long review of Age of Em (9K words, 6% as long as the book), wherein he not only likes and recommends it, he also accepts pretty much all its claims within its main focus. That is, I present my book as being expert on the topic of what would actually happen if cheap ems were our next huge social change. Where Alexander disagrees is on two auxiliary topics, which I mention but on which I claim less expertise, namely how likely is this key scenario assumption, and how valuable is the resulting civilization I describe.

On the subject of value, Alexander leans forager (i.e., liberal) on the forager vs. farmer scale. He dislikes civilization evolving away from the behaviors and values of our forager ancestors, and today he partly blames this on capitalism. He doesn’t see our increase in numbers, comfort, and lifespan as sufficient compensation. (I think he’d like the book Against Civilization.) He says:

[Nick Land’s Ascended Economy] seems to me the natural end of the economic system. Right now it needs humans only as laborers, investors, and consumers. But robot laborers are potentially more efficient, companies based around algorithmic trading are already pushing out human investors, and most consumers already aren’t individuals – they’re companies and governments and organizations. At each step you can gain efficiency by eliminating humans, until finally humans aren’t involved anywhere. .. The Age of Em is an economy in the early stages of such a transformation. Instead of being able to replace everything with literal robots, it replaces them with humans who have had some aspects of their humanity stripped away. Biological bodies. The desire and ability to have children normally. ..

I envision a spectrum between the current world of humans and Nick Land’s Ascended Economy. Somewhere on the spectrum we have ems who get leisure time. A little further on the spectrum we have ems who don’t get leisure time. But we can go further. .. I expect [greatly reduced sex desire] would happen about ten minutes after the advent of the Age of Em .. Combine that with the stimulant use mentioned above, and you can have people who will never have nor want to have any thought about anything other than working on the precise task at which they are supposed to be working at any given time. ..

I see almost no interesting difference between an em world with full use of these tweaks and an Ascended Economy world. Yes, there are things that look vaguely human in outline laboring in the one and not the other, but it’s not like there will be different thought processes or different results. I’m not even sure what it would mean for the ems to be conscious in a world like this – they’re not doing anything interesting with the consciousness. .. If we get ems after all, I expect them to be lobotomized and drugged until they become effectively inhuman, cogs in the Ascended Economy that would no more fall in love than an automobile would eat hay and whinny.

Alexander seems to strongly endorse the usual forager value of leisure over work, so much so that he can’t see people focused on their work as human, conscious, or of any moral value. Creatures only seem valuable to him to the extent that they have sex, leisure time, minds wandering away from work, and desires to do things other than work.

This seems ironic because Scott Alexander is one of the most human and productive workers I know. He has a full time job as a psychiatrist, an especially demanding job, and in addition finds time to write frequent long careful analyses of many topics. I find it hard to see where he has that much time for leisure, and doubt he would in fact be substantially more productive overall if he took drugs to make him forget sex, mentally wander less, and focus more on his immediate tasks. He is exactly the sort of person an em economy would want many copies of, pretty much just as he is. Yet if we are to believe him, he only sees value in his brief leisure hours.

I see Alexander as having too little respect for the functionality of human behaviors and mind design. Yes, maximally competitive em-era behaviors and minds won’t be exactly like current ones. But that doesn’t necessarily mean one wants to throw out most existing behaviors and brain modules wholesale and start over from scratch. As these behaviors and modules all arose because they helped our ancestors be more competitive in some prior context, it makes more sense to try to repair, reform, and repurpose them.

For example, the robust productivity gains observed from workers who take breaks don’t seem to depend much on worker motivation. Breaks aren’t just about motivation; they are a deeply entrenched part of being productive. Similarly, wandering minds may take away from the current immediate task, but they help one to search for hidden problems and opportunities. Also, workers today who focus on just doing immediate tasks often lose out to others who attend more to building and managing social relations, as well as office politics. Love and sex can be very helpful in forming and maintaining relations.

Of course I’m not trying to offer any long term assurances, and it is quite reasonable to worry about what we will lose along with what we will gain. But since today most of the people we most respect and celebrate tend to be workaholics, I just can’t buy the claim that most of us today can’t find value in similarly productive and work-focused ems. And I just can’t see thoughtless workers being the most productive in the early em era of my book.

GD Star Rating
loading...
Tagged as: , ,

Rating Ems vs AIs

Everyone without exception believes his own native customs, and the religion he was brought up in, to be the best (Herodotus 440BC).

I’ve given about sixty talks so far on the subject of my book The Age of Em. A common response is to compare my scenario to one where instead of ems, it is non-emulation-based software that can first replace humans on most all jobs. While some want to argue about which tech may come first, most prefer to evaluate which tech they want to come first.

Most who compare to ems to non-em-AI seem to prefer the latter. Some say they are concerned because they see ems as having a lower quality of life than we do today (more on that below). But honestly I mostly hear about humans losing status. Even though both meat humans and ems can both be seen as our descendants, people identify more with meat as “us” and see ems as “them.” So they lament meat no longer being the top dog in-charge center-of-attention.

The two scenarios have many similarities. In both scenarios, meat humans must all retire, and robots take over managing the complex details of this new world, which humans are too slow, distant, and stupid to manage. The world economy can grow very fast, letting meat get collectively very rich, and which meat soon starves depends mostly on how well meat insures and shares among themselves. But it is hard to offer much assurance of long run stability, as the world can plausibly change so fast.

Ems, however, seem more threatening to status than other kinds of sophisticated capable machinery. You can more vividly imagine ems more clearly winning the traditional contests whereby humans compete for status, and then afterward acting superior, such as by laughing at meat humans. In contrast, other machines can be so alien that we may not be tempted to make status comparisons with them.

If, in contrast, your complaint about the em world is that ems have a lower quality of life, then you have to either care about something more like an average quality of life, or you have to argue that the em quality of life is below some sort of “zero”, i.e., the minimum required for a life to be worth living (or having existed). And this seems to me a hard case to make.

Oh I can see you thinking that em lives aren’t as good as yours; pretty much all cultures find ways to see their culture as superior. But unless you argue that em lives are much worse than the typical human life in history, then either you must say the typical human life was not worth living, or you must accept em lives as worth living. And if you claim that the main human lives that have been worth living are those in your culture, I’ll shake my head at your incredible cultural arrogance.

(Yes, some like Nick Bostrom in Superintelligence, focus on which scenario reduces existential risk. But even he at one point says “On balance, it looks like the risk of an AI transition would be reduced if whole brain emulation comes before AI,” and in the end he can’t seem to rank these choices.)

GD Star Rating
loading...
Tagged as: , ,

4 Age of Em Web Reviews

In addition to the Financial Times review I posted on, there are now four web reviews of Age of Em. (Kindle version now $14.39, delivers tomorrow.) In reverse chronological order:

Peter McCluskey, The Age of Em, Bayesian Investor, May 24.

This book analyzes a possible future era when software emulations of humans (ems) dominate the world economy. It is too conservative to tackle longer-term prospects for eras when more unusual intelligent beings may dominate the world. Hanson repeatedly tackles questions that scare away mainstream academics, and gives relatively ordinary answers. .. Even if we should expect a much less than 50% chance of Hanson’s scenario becoming real, it seems quite valuable to think about how comfortable we should be with it and how we could improve on it.

Neerav Kingsland, Book Review: The Age of Em, relinquishment, May 24.

It is worth reading for (at least) the following reasons: It is a survey of  us: To extrapolate what the em world might be like, Robin summarize the key findings of numerous academic fields. This vehicle makes the book a fascinating survey of what we know about humans – think David Brook’s The Social Animal, but through a more removed lens, and with a deeper blend of hard and soft sciences. It might not be too far away: .It is a thoughtful framework: ..

Peter St Onge, Review: Robin Hanson’s Age of Em, Profits of Chaos, May 22.

If you know Hanson you won’t be surprised that he packs a lot of ideas into one book. The pacing is fast, chock-full of interesting ideas to play with. .. Hanson has done a fantastic job sketching technologically and economically plausible outcomes to the future of humans and near-humans. He’s achieved what he set out to do, to stimulate these discussions, bring more minds into the game. I’d thoroughly recommend a read if you’re even remotely interested in the future of technology and humanity.

Gaspar, review, Goodreads, April 13.

It seems he is just another victim of the conjunction fallacy. I still find hard to conceive how “today’s standard academic consensus science” (as the author writes) could analyze in an accurate and realistic way His very specific vision of future. .. I believe that some of the author’s statements (or maybe his way of expressing them) are very arrogant. .. All in all, it is a very interesting intellectual exercise to use today consensus theories from many fields to try to understand or imagine some possible future among many other possible futures.

This last review doesn’t offer any specific rebuttals, and was so early it must have been based on a draft. But it expresses a reaction I’ve seen often at my talks: incredulity that anyone could think such an analysis possible. Kingsland asks similarly:

While it seems clear that humans can make decent forecasts within their singularity (Robin gives some examples), it’s another to think that humans can make decent forecasts across singularities. Could a hunter and gatherer really have predicted the  industrial world? A farmer?

But we can use today’s social, human, and physical sciences to understand past farming and foraging eras in some detail, suggesting that such sciences do apply beyond our era. However, as David Lewis famously quipped, “I cannot refute an incredulous stare.”

St Onge disagrees with my estimate of near subsistence em wages; he estimates them to be a ten billion times larger than subsistence “because costs per life of building and maintaining their world will be so low.” I guess he just can’t imagine very low marginal productivity. Kingsland suggests culture might raise wages:

Perhaps it would become taboo to replicate yourself, akin to teenage pregnancy?

But if the fraction of ems willing to copy freely was anything like current, or even ancient, teen pregnancy rates, em wages must fall low.

McCluskey wants me to make one estimate more precise:

Wages in this scenario are somewhat close to subsistence levels. Ems have some ability to restrain wage competition, but less than they want. Does that mean wages are 50% above subsistence levels, or 1%? Hanson hints at the former. The difference feels important to me. I’m concerned that sound-bite versions of book will obscure the difference.

I’ll admit that I didn’t estimate this because I didn’t feel like I had a strong enough basis. But on reflection I’d guess that the ratio of productivity between the 90th and 10th percentile moves from today’s factor of 2 in US, and factor of 5 in the developing world, to more like a factor of 1.1 or 1.2. That might give the typical em roughly two extra hours of time per day to spend as they like, most of which ems plausibly spend on vying for status, perhaps at work.

GD Star Rating
loading...

Problem, No Solution Taboo?

Three years ago I described the “What if Failure Taboo”:

A simple moral principle: when a future change is framed as a problem which we might hope our political system to solve, then the only acceptable reason to talk about the consequences of failing to solve that problem is to scare folks into trying harder to solve it. If you instead assume that politics will fail to solve the problem, and analyze the consequences of that in more detail, not to scare people but to work out how to live in that scenario, you are seen as expressing disloyalty to the system and hostility toward those who will suffer from that failure.

I suggested this could be an issue with my book Age of Em:

All of which seems bad news for my book, which mostly just accepts the “robots take over, humans lose wages and get sidelined” scenario and analyzes its consequences. No matter how good my reasons for thinking politics will fail to prevent this, many will react as did Nikola Danaylov, with outrage at my hostility toward the poor suffering losers.

This week I talked on my book to a sharp lively group organized by Azeem Azhar (author of the futurist newsletter Exponential View), and learned that this taboo may be worse than I thought. I tried to present the situation as something that you might consider to be a problem, but that while my analysis should enable better problem solving, I’ve personally focused on just describing this situation. Mixing up normative and positive discussions risks the positive being overshadowed by the normative, and positive claims seeming less reliable when mixed up with more disputable normative claims.

Even with this reframing, several people saw me as still violating the key taboo. Apparently it isn’t just taboo to assume that we’ll fail to solve a problem; it can also be taboo to merely describe a problem without recommending a solution. At least when the problem intersects with many strong feelings and moral norms. To many, neutral analysis just seems cold and uncaring, and suspiciously like evil.

GD Star Rating
loading...
Tagged as: ,

Astonishing Questions Exist!

An astonishing claim is one that, if true, has some sort of surprising hard-to-accept implication. You stare at that implication, and wow, who would have believed that? For example, it would be astonishing to me to win the Nobel Prize. An astonishing question is one where most every possible answer is an astonishing claim. And an astonishing fact about our world is that we know of many astonishing questions.

For example, consider the question: does the universe go back forever in time, or was there a first time? Either way, something seems amazing. Really, you just keep going back in time and things just keep getting simpler with lower entropy, forever? Or, really, there is this moment in time and space-time just ends there?

Analogous astonishing questions exist on if the universe goes on forever in the wide future, or as a future path goes a black hole. And when we’ve looked inside things, we’ve always found more detail so far. Is there really a level of detail where we never ever find anything inside that? Or will there come a time in the future where innovation and discovery runs out, so that we stop learning and inventing new things? Will growth rates keep accelerating, or slow down forever? Will our descendants die out or last forever? Will they be as different from us as we are from our distant ancestors, or will they stay like us forever?

Will we eventually contact real advanced aliens? Will we develop immortality? Is stable world peace possible, or effective world governance? Will we learn to effectively share info so that we don’t knowingly disagree, or will we continue forever to irrationally disagree? Will it eventually be possible to make time machines, or will that be forever impossible? Will it become possible to write software that is as smart as humans across the board? Will we become able to upload/emulate human brains on computers? Do humans have free will? Do human brains have some special capacity for consciousness, or is all matter capable of consciousness?

Now in a literal sense the following question is also astonishing: who will win the lottery? That is, for each possible lottery winner, if we focus on them it seem astonishing to think: they won the lottery! Yet of course we know that someone must win. But the above questions seem much more astonishing to me in some way, as they each seem to require a more dramatic revision of my view of something more substantial. Somehow, in ways we find it hard to accept now, we will be astonished!

The existence of astonishing questions makes it dangerous to rely on the following form of argument: It would be astonishing if X were not true, therefore probably X. Seems like a safe solid argument, but it isn’t.

GD Star Rating
loading...

Me in London, Cambridge

Over the next week I’ll give these talks on Age of Em:

I’ll also talk in Paris May 18, but that is by invitation only.

Added 15May: Ebook versions are now available for pre-order.

GD Star Rating
loading...
Tagged as:

FT Reviews Age of Em

The first MSM review is in for Age of Em. It is by Sarah O’Connor (really!), in the Financial Times. She “gets” me:

Plenty of futurists and science fiction writers have toyed with the idea that the brains of particular humans could one day be scanned and “uploaded” into artificial hardware but Prof Hanson’s take is different. His aim is to use standard theories from the physical, human and social sciences to make forecasts about how this technological breakthrough would really change our world. ..

Some of the most profound questions .. are the questions on which Prof Hanson is most tentative and brief because he has set out deliberately to focus on the predictable and to avoid being “creative or contrarian”. This is reasonable enough and the book succeeds on its own terms. Still, it does leave a hole. Perhaps it will fall to the science fiction writers to fill it.

Yes, mine are first steps, and I hope others will further develop this scenario. O’Connor does have a few words of mild criticism:

The book is crammed full of such fascinating visions of an imagined future. Still, some readers will share criticisms the author says he has encountered. For any of this to seem plausible, one has to believe that we will invent the ability and be willing to scan and copy human brains. Not everyone will accept this.

One also has to believe that current economic and social theories will hold in this strange new world; that the “unknown unknowns” are not so great as to make any predictions impossible. Certainly, some of the forecasts seem old-fashioned, like the notion that male ems will prefer females with “signs of nurturing inclinations and fertility, such as youthful good looks” while females will prefer males with “signs of wealth and status”.

Yes, we can’t be at all certain of this scenario. But if it is worth having a hundred books on future scenarios, it is worth having books that analyze scenarios with only a 1% chance of happening.

Yes, you have to think that social scientists, like physicists, engineers, and computer scientists, actually know some generalities not hopelessly tied to the details of our particular time and culture. Such as our standard results on robust “old-fashioned” sex differences in long-term mate preferences:

Several standard sex differences replicated across cultures, including womens greater valuation of social status and mens greater valuation of physical attractiveness. (more; see also, also, also)

Even if such differences were weaker in nations with greater gender parity, they’d still remain. (But in fact such differences actually seem stronger there.) Yes, such differences may be in part caused by culture. But whatever their cause, they seem pervasive and robust enough to make them likely to continue on in an em world.

GD Star Rating
loading...
Tagged as:

Lognormal Jobs

I often meet people who think that because computer tech is improving exponentially, its social impact must also be exponential. So as soon as we see any substantial social impact, watch out, because a tsunami is about to hit. But it is quite plausible to have exponential tech gains translate into only linear social impact. All we need is a lognormal distribution, as in this diagram:

LogNormalJobs

Imagine that each kind of jobs that humans do requires a particular level of computing power in order for computers to replace humans on that job. And imagine that these job power levels are distributed lognormally.

In this case an exponential growth in computing power will translate into a linear rate at which computers displace humans on jobs. Of course jobs may clump along this log-computing-power axis, giving rise to bursts and lulls in the rate at which computers displace jobs. But over the long run we could see a relatively steady rate of job displacement even with exponential tech gains. Which I’d say is roughly what we do see.

Added 3am: Many things are distributed lognormally.

GD Star Rating
loading...
Tagged as: , ,

The Labor-From-Factories Explosion

As I’ve discussed before, including in my book, the history of humanity so far can be roughly summarized as a sequence of three exponential growth modes: foragers with culture started a few million years ago, farming started about ten thousand years ago, and industry starting a few hundred years ago. Doubling times got progressively shorter: a quarter million years, then a millennia, and now fifteen years. Each time the transition lasted less than a previously doubling time, and roughly similar numbers of humans have lived during each era.

Before humans, animal brains brains grew exponentially, but even more slowly, doubling about every thirty million years, starting about a half billion years ago. And before that, genomes seem to have doubled exponentially about every half billion years, starting about ten billion years ago.

What if the number of doublings in the current mode, and in the mode that follows it, are comparable to the number of doublings in the last few modes? What if the sharpness of the next transition is comparable to the sharpness if the last few transitions, and what if the factor by which the doubling time changes next time is comparable to the last few factors. Given these assumptions, the next transition will happen sometime in roughly the next century. Within a period of five years, the economy will be doubling every month or faster. And that new mode will only last a year or so before something else changes.

To summarize, usually in history we see relatively steady exponential growth. But five times so far, steady growth has been disturbed by a rapid transition to a much faster rate of growth. It isn’t crazy to think that this might happen again.

Plausibly, new faster exponential modes appear when a feedback loop that was previously limited and blocked becomes is unlocked and strong. And so one way to think about what might cause the next faster mode after ours is to look for plausible feedback loops. However, if there thousands of possible factors that matter for growth and progress, then there are literally millions of possible feedback loops.

For example, denser cities should innovate more, and more innovation can find better ways to make buildings taller, and thus increase city density. More better tutorial videos make it easier to learn varied skills, and some of those skills help to make more better tutorial videos. We can go all day making up stories like these.

But as we have only ever seen maybe five of these transitions in all of history, powerful feedback loops whose unlocking causes a huge growth rate jump must be extremely rare. The vast majority of feedback loops do not create such a huge jump when unlocked. So just because you can imagine a currently locked feedback loop does not make unlocking it likely to cause the next great change.

Many people lately have fixated on one particular possible feedback loop: an “intelligence explosion.”  The more intelligence a creature is, the more it is able to change creatures like itself to become more intelligent. But if you mean something more specific than “mental goodness” by “intelligence”, then this remains only one of thousands of possibilities. So you need strong additional arguments to see this feedback loop as more likely than all the others. And the mere fact that you can imagine this feedback being positive is not remotely enough.

It turns out that we already know of an upcoming transition of a magnitude similar to the previous transitions, scheduled to arrive roughly when prior trends led us to expect a new transition. This explosion is due to labor-from-factories.

Today we can grow physical capital very fast in factories, usually doubling capital on a scale ranging from a few weeks to a few months, but we grow human workers much more slowly. Since capital isn’t useful without more workers, we are forced to grow today mainly via innovation. But if in the future we find a way to make substitutes for almost all human workers in factories, the economy can grow much faster. This is called an AK model, and standard growth theory says it is plausible that this could let the economy double every month or so.

So if it is plausible that artificial intelligence as capable as humans will appear in the next century or so, then we already know what will cause the next great jump to a faster growth mode. Unless of course some other rare powerful feedback loop is unlocked before then. But if an intelligence explosion isn’t  possible until you have machines at least as smart as humans, then that scenario won’t happen until after labor-from-factories. And even then it is far from obvious that feedback can cause one of the few rare big growth rate jumps.

GD Star Rating
loading...
Tagged as: , , ,

Me at BGSU, CMU, MIT

While last week I talked at U Rochester, the next three weeks I talk at:

All these talks are, of course, on my upcoming book The Age of Em.

GD Star Rating
loading...
Tagged as: