Tag Archives: Ems

Rating Ems vs AIs

Everyone without exception believes his own native customs, and the religion he was brought up in, to be the best (Herodotus 440BC).

I’ve given about sixty talks so far on the subject of my book The Age of Em. A common response is to compare my scenario to one where instead of ems, it is non-emulation-based software that can first replace humans on most all jobs. While some want to argue about which tech may come first, most prefer to evaluate which tech they want to come first.

Most who compare to ems to non-em-AI seem to prefer the latter. Some say they are concerned because they see ems as having a lower quality of life than we do today (more on that below). But honestly I mostly hear about humans losing status. Even though both meat humans and ems can both be seen as our descendants, people identify more with meat as “us” and see ems as “them.” So they lament meat no longer being the top dog in-charge center-of-attention.

The two scenarios have many similarities. In both scenarios, meat humans must all retire, and robots take over managing the complex details of this new world, which humans are too slow, distant, and stupid to manage. The world economy can grow very fast, letting meat get collectively very rich, and which meat soon starves depends mostly on how well meat insures and shares among themselves. But it is hard to offer much assurance of long run stability, as the world can plausibly change so fast.

Ems, however, seem more threatening to status than other kinds of sophisticated capable machinery. You can more vividly imagine ems more clearly winning the traditional contests whereby humans compete for status, and then afterward acting superior, such as by laughing at meat humans. In contrast, other machines can be so alien that we may not be tempted to make status comparisons with them.

If, in contrast, your complaint about the em world is that ems have a lower quality of life, then you have to either care about something more like an average quality of life, or you have to argue that the em quality of life is below some sort of “zero”, i.e., the minimum required for a life to be worth living (or having existed). And this seems to me a hard case to make.

Oh I can see you thinking that em lives aren’t as good as yours; pretty much all cultures find ways to see their culture as superior. But unless you argue that em lives are much worse than the typical human life in history, then either you must say the typical human life was not worth living, or you must accept em lives as worth living. And if you claim that the main human lives that have been worth living are those in your culture, I’ll shake my head at your incredible cultural arrogance.

(Yes, some like Nick Bostrom in Superintelligence, focus on which scenario reduces existential risk. But even he at one point says “On balance, it looks like the risk of an AI transition would be reduced if whole brain emulation comes before AI,” and in the end he can’t seem to rank these choices.)

GD Star Rating
loading...
Tagged as: , ,

Problem, No Solution Taboo?

Three years ago I described the “What if Failure Taboo”:

A simple moral principle: when a future change is framed as a problem which we might hope our political system to solve, then the only acceptable reason to talk about the consequences of failing to solve that problem is to scare folks into trying harder to solve it. If you instead assume that politics will fail to solve the problem, and analyze the consequences of that in more detail, not to scare people but to work out how to live in that scenario, you are seen as expressing disloyalty to the system and hostility toward those who will suffer from that failure.

I suggested this could be an issue with my book Age of Em:

All of which seems bad news for my book, which mostly just accepts the “robots take over, humans lose wages and get sidelined” scenario and analyzes its consequences. No matter how good my reasons for thinking politics will fail to prevent this, many will react as did Nikola Danaylov, with outrage at my hostility toward the poor suffering losers.

This week I talked on my book to a sharp lively group organized by Azeem Azhar (author of the futurist newsletter Exponential View), and learned that this taboo may be worse than I thought. I tried to present the situation as something that you might consider to be a problem, but that while my analysis should enable better problem solving, I’ve personally focused on just describing this situation. Mixing up normative and positive discussions risks the positive being overshadowed by the normative, and positive claims seeming less reliable when mixed up with more disputable normative claims.

Even with this reframing, several people saw me as still violating the key taboo. Apparently it isn’t just taboo to assume that we’ll fail to solve a problem; it can also be taboo to merely describe a problem without recommending a solution. At least when the problem intersects with many strong feelings and moral norms. To many, neutral analysis just seems cold and uncaring, and suspiciously like evil.

GD Star Rating
loading...
Tagged as: ,

FT Reviews Age of Em

The first MSM review is in for Age of Em. It is by Sarah O’Connor (really!), in the Financial Times. She “gets” me:

Plenty of futurists and science fiction writers have toyed with the idea that the brains of particular humans could one day be scanned and “uploaded” into artificial hardware but Prof Hanson’s take is different. His aim is to use standard theories from the physical, human and social sciences to make forecasts about how this technological breakthrough would really change our world. ..

Some of the most profound questions .. are the questions on which Prof Hanson is most tentative and brief because he has set out deliberately to focus on the predictable and to avoid being “creative or contrarian”. This is reasonable enough and the book succeeds on its own terms. Still, it does leave a hole. Perhaps it will fall to the science fiction writers to fill it.

Yes, mine are first steps, and I hope others will further develop this scenario. O’Connor does have a few words of mild criticism:

The book is crammed full of such fascinating visions of an imagined future. Still, some readers will share criticisms the author says he has encountered. For any of this to seem plausible, one has to believe that we will invent the ability and be willing to scan and copy human brains. Not everyone will accept this.

One also has to believe that current economic and social theories will hold in this strange new world; that the “unknown unknowns” are not so great as to make any predictions impossible. Certainly, some of the forecasts seem old-fashioned, like the notion that male ems will prefer females with “signs of nurturing inclinations and fertility, such as youthful good looks” while females will prefer males with “signs of wealth and status”.

Yes, we can’t be at all certain of this scenario. But if it is worth having a hundred books on future scenarios, it is worth having books that analyze scenarios with only a 1% chance of happening.

Yes, you have to think that social scientists, like physicists, engineers, and computer scientists, actually know some generalities not hopelessly tied to the details of our particular time and culture. Such as our standard results on robust “old-fashioned” sex differences in long-term mate preferences:

Several standard sex differences replicated across cultures, including womens greater valuation of social status and mens greater valuation of physical attractiveness. (more; see also, also, also)

Even if such differences were weaker in nations with greater gender parity, they’d still remain. (But in fact such differences actually seem stronger there.) Yes, such differences may be in part caused by culture. But whatever their cause, they seem pervasive and robust enough to make them likely to continue on in an em world.

GD Star Rating
loading...
Tagged as:

The Labor-From-Factories Explosion

As I’ve discussed before, including in my book, the history of humanity so far can be roughly summarized as a sequence of three exponential growth modes: foragers with culture started a few million years ago, farming started about ten thousand years ago, and industry starting a few hundred years ago. Doubling times got progressively shorter: a quarter million years, then a millennia, and now fifteen years. Each time the transition lasted less than a previously doubling time, and roughly similar numbers of humans have lived during each era.

Before humans, animal brains brains grew exponentially, but even more slowly, doubling about every thirty million years, starting about a half billion years ago. And before that, genomes seem to have doubled exponentially about every half billion years, starting about ten billion years ago.

What if the number of doublings in the current mode, and in the mode that follows it, are comparable to the number of doublings in the last few modes? What if the sharpness of the next transition is comparable to the sharpness if the last few transitions, and what if the factor by which the doubling time changes next time is comparable to the last few factors. Given these assumptions, the next transition will happen sometime in roughly the next century. Within a period of five years, the economy will be doubling every month or faster. And that new mode will only last a year or so before something else changes.

To summarize, usually in history we see relatively steady exponential growth. But five times so far, steady growth has been disturbed by a rapid transition to a much faster rate of growth. It isn’t crazy to think that this might happen again.

Plausibly, new faster exponential modes appear when a feedback loop that was previously limited and blocked becomes is unlocked and strong. And so one way to think about what might cause the next faster mode after ours is to look for plausible feedback loops. However, if there thousands of possible factors that matter for growth and progress, then there are literally millions of possible feedback loops.

For example, denser cities should innovate more, and more innovation can find better ways to make buildings taller, and thus increase city density. More better tutorial videos make it easier to learn varied skills, and some of those skills help to make more better tutorial videos. We can go all day making up stories like these.

But as we have only ever seen maybe five of these transitions in all of history, powerful feedback loops whose unlocking causes a huge growth rate jump must be extremely rare. The vast majority of feedback loops do not create such a huge jump when unlocked. So just because you can imagine a currently locked feedback loop does not make unlocking it likely to cause the next great change.

Many people lately have fixated on one particular possible feedback loop: an “intelligence explosion.”  The more intelligence a creature is, the more it is able to change creatures like itself to become more intelligent. But if you mean something more specific than “mental goodness” by “intelligence”, then this remains only one of thousands of possibilities. So you need strong additional arguments to see this feedback loop as more likely than all the others. And the mere fact that you can imagine this feedback being positive is not remotely enough.

It turns out that we already know of an upcoming transition of a magnitude similar to the previous transitions, scheduled to arrive roughly when prior trends led us to expect a new transition. This explosion is due to labor-from-factories.

Today we can grow physical capital very fast in factories, usually doubling capital on a scale ranging from a few weeks to a few months, but we grow human workers much more slowly. Since capital isn’t useful without more workers, we are forced to grow today mainly via innovation. But if in the future we find a way to make substitutes for almost all human workers in factories, the economy can grow much faster. This is called an AK model, and standard growth theory says it is plausible that this could let the economy double every month or so.

So if it is plausible that artificial intelligence as capable as humans will appear in the next century or so, then we already know what will cause the next great jump to a faster growth mode. Unless of course some other rare powerful feedback loop is unlocked before then. But if an intelligence explosion isn’t  possible until you have machines at least as smart as humans, then that scenario won’t happen until after labor-from-factories. And even then it is far from obvious that feedback can cause one of the few rare big growth rate jumps.

GD Star Rating
loading...
Tagged as: , , ,

Is Forgotten Party Death?

Consider 3 possible space-time experience trajectories:

ForgottenPartyToSpurIn A, a person takes a drug at the start of a party which causes them to not remember that party the next day or anytime later. In C, an em splits off a short term “spur” copy which does a short term task and then ends.

Scenario B can be seen as like A, except that right after taking the drug the person quickly moves to a distant party location. Call this B1.  Alternatively, B can be seen as like C, except that the em original is archived and inactive while the spur works. Call this B2.

In my talks on Age of Em, I’ve heard many object to scenario C, but few object to A. In scenario A, few think they’d be stressed near the end of the party, thinking they are about to “die.” Yet many see scenario C as a “death,” and claim the em spur would refuse to do their assigned task, and instead fight fiercely to keep going.

Scenario B is designed to be intermediate between A and B, and so to force a choice between conflicting intuitions. Would you really see B2 as “death” but B1 as no big deal, even though they have the same space time structure of experiences? If not, I think you should admit either that A is “death”, or that C is not. Or explain what matters besides the space-time structure of experience.

I confidently predict that ems see all of these as no big deal, because a competitive em world selects for ems who are more productive, and a willingness to create short term spurs is quite productive in the em world.

Note that this issue engages similar space vs time morality intuitions as these three prior posts.

GD Star Rating
loading...
Tagged as: , ,

My Circus Sideshow

If a fossil of an an alien, or any alien artifact, were put it on display, it would attract millions. Sure some would see it because of its objective importance. But most would come just because it is weird.

People used to see a traditional circus sideshow for similar reasons. But consider: once you know that there exist dwarves, sword swallowers, and women with beards, what do you learn more by seeing them person? Yes, in part you just want to brag about how much you’ve seen, but you are also actually curious about what such things look like up close.

Circus side shows are weird, but they are also far from maximally strange. Many ocean creatures are far stranger. The attraction is in part a mixture of the strange and familiar. Once a familiar thing has changed in one very big way, one naturally wonders what other aspects of it are changed and how. One doesn’t wonder that about something where all its features are strange.

Tyler Cowen suggests this as the appeal of my upcoming book The Age of Em: Work, Love, and Life When Robots Rule the Earth:

The ostensible premise of the book is that people have become computer uploads, and we have an entirely new society to think about: how it works, what problems it has, and how it evolves. .. But this is more than just a nerdy tech book, it is also:

  • Straussian commentary on the world we actually live in. ..
  • A reminder of how strange everything is .. It’s a mock of all those who believe in individual free will.
  • An attempt to construct a fully rational theology ..
  • An extended essay on the impossibility of avoiding theology ..
  • A satire on the rest of social science, and how we try to explain and predict the future.
  • A meta-level growth model in which energy alone matters and the “fixed factor” assumptions of other models are relativized. ..
  • A challenge to our notions of wherein the true value of a life resides. (more)

I describe an entire world in great detail, a world that is a mix between a strange alien civilization and our familiar world. Any world described in enough detail must raise issues that look like theology, including free will and where true value resides. And any detailed strange yet familiar world can be seen as satire on social science and Straussian commentary on our world.

So the key is that, like a circus side show, my book lets readers see something strange yet familiar in great detail, so they can gawk at what else changes and how when familiar things change. My book is a dwarf, sword swallower, and bearded lady, writ large.

Okay, yeah, I can accept that as the main appeal of my book. Just as the main appeal of seeing an alien fossil to most would be its strangeness. Even if understanding aliens were actually vitally important.

GD Star Rating
loading...
Tagged as: ,

Here Be Dragons

In his new book Here Be Dragons: Science, technology, and the future of humanity, Olle Haggstrom mostly discusses abstract and philosophical issues. But at one point in the book he engages the more specific forecasts I discuss in my upcoming book. So let me quote him and offer a few responses:

Once successful whole-brain emulation has been accomplished, it might not be long before it becomes widely available and widely-used. This bring us to question (4) – what will society be like when uploading is widely available? Most advocates of an uploaded posthuman existence, such as Kurzweil and Goertzel, point at the unlimited possibilities for an unimaginably (to us) riche and wonderful life in ditto virtual realities. One researcher stands out from the rest in actually applying economic theory and social science to attempt to sketch how a society of uploads will turn out is the American economist Robin Hanson, beginning in a 1994 paper, continuing with a series of posts on his extraordinary blog Overcoming Bias, and summarizing his findings (so far) in a chapter in Intelligence Unbound and in an upcoming book.

Two basic assumptions for Hanson’s social theory of uploads are
(i) that whole-brain emulation is achieved mostly by brute force, with relatively little scientific understanding of how thoughts and other high-level phenomena supervene on the lower-level processes that are simulated, and
(ii) that current trends of hardware costs decreasing at a fast exponential rate will continue (if not indefinitely then at least far into the era he describes).

Actually, I just need to assume that at some point the hardware cost is low enough to make uploads substantially cheaper than human workers. I don’t need to make assumptions about rates at which hardware costs fall.

Assumption (i) prevents us from boosting the emulated minds to superhuman intelligence levels, other than in terms of speed, by transferring the mot faster hardware. Assumption (ii) opens up the possibility for quickly populating the world with billions and then trillions of uploaded minds, which is in fact what Hanson predicts will happen. ..

Actually, population increases quickly mainly because factories can crank out an amount of hardware equal to their own economic value in a short time – months or less.

Decreases in hardware costs will push down wages. .. This will send society to the classical Malthusian trap in which population will grown until it is hit by starvation (uploaded minds will not need food, of course, but things like energy, CPU time and disk space). ..

There are many exotica in Hanson’s future. One is that uploads can fun on different hardware and thus at different speeds, depending on circumstances. .. Even more exotic is the idea that most work will be done by short-lived so-called spurs, copied from a template upload to work for, say, a few hours and then be terminated (i.e., die). .. Will they not revolt? The question has been asked, but Hanson maintains that “when life is cheap, death is cheap.”

First, spurs could retire to a much slower speed instead of ending. Second, just before an em considers whether to split off a spur copy for a task, that em can ask itself if it would be willing to do that assigned task if it found itself a few seconds later to be the spur. Ems should quickly learn to reliable estimate their own willingness, so they just won’t split off spurs if they estimated a high chance that the spur would become troublesome. Maybe today we find it hard to estimate such things, but they’d know their world well so it would an easy question for them. So I just can’t see spur rebellion as a big practical problem, any more than we have a big problem planning to go to work for the day and then suddenly going to the movies instead.

The future outlined in Hanson’s theory of uploaded minds may seem dystopian .. but Hanson does not accept this label, and his main retorts seem to be twofold. First, population numbers will be huge, which is good if we accept that the value of a future should be measured .. by the total amount of well-being, which in a huge population can be very large even if each individual has only a modest positive level of well-being. Second, the trillions of short-lived uploaded minds working hard for their subsistence right near starvation level can be made to enjoy themselves, e.g., by cheap artificial stimulation of their pleasure center.

I don’t think I’ve ever talked about “cheap artificial stimulation of their pleasure center.” I instead say that most ems work and leisure in virtual worlds of spectacular quality, and that ems need never experience hunger, disease, or intense pain, nor ever see, hear, feel, or taste grime or anything ugly or disgusting. Yes they’d work most of the time but their jobs would be mentally challenging, they’d be selected for being very good at their jobs, and people can find deep fulfillment in such modes. We are very culturally plastic, and em culture would promote finding value and fulfillment in typical em lives. In addition, I estimate that most humans who have ever lived have had lives worth living, in part because of this cultural plasticity.

Then there’s the issue of whether and to what extent we should view Hanson’s analysis as a trustworthy prediction of what will actually happen. A healthy load of skepticism seems appropriate. .. It also seems that he works so far outside of the comfort zones of where economic theory has been tested empirically, and uses so many explicit and implicit assumptions that are open to questioning, that his scenarios need to be taken with a grain of salt (or a full bushel).

You could say this about any theoretical analysis of anything not yet seen. All theory requires you to make assumptions, and all assumptions are open to questioning. Perhaps my case is worse than others, but the above certainly doesn’t show that to be the case.

One obvious issue to consider is whether society following a breakthrough in the technology will be better or worse than society without such a breakthrough. The utopias hinted at by, e.g., Kurzweil and Goertzel seem pretty good, whereas Hanson’s Malthusian scenario looks rather less appealing.

But Kurzweil and Goertzel offer inspiring visions, not hard-headed social science analysis. Of course that will sound better.

GD Star Rating
loading...
Tagged as:

Age of Em in Amsterdam

At 6pm on Tuesday, 24 November 2015, I’ll speak at Amsterdam University College on:

The Age of Em: Work, Love and Life when Robots Rule the Earth

Robots may one day rule the world, but what is a robot-ruled earth like? Many think the first truly smart robots will be brain emulations or ems. Scan a human brain, then run a model with the same connections on a fast computer and you have a robot brain, but recognisably human. Ems make us question common assumptions of moral progress because they reject many of the values we hold dear. Applying decades of expertise in physics, computer science and economics, Robin Hanson uses standard theories to paint a detailed picture of a world dominated by ems. (more)

The day before I’ll speak on the same subject at an invitation-only session of CIO Day. Added: I’ll also be on a panel on Enterprise Prediction Markets during the more open session on Tuesday.

GD Star Rating
loading...
Tagged as: , ,

News of What?

Today’s New York Times has a 7000 word article by Amy Harmon on cryonics, brain scanning, and brain emulation. Now these are subjects of great interest to me; my first book comes out in spring on the third topic. And 7000 words is space to say a great deal, even if you add the constraint that what you say must be understandable to the typical NYT reader.

So I’m struck by the fact that I have almost nothing to say in response to anything particular said in this article. Ms. Harmon gives the most space to one particular young cryonics patient who got others to donate to pay for her freezing. This patient hopes to return via brain emulation. Ms. Harmon discusses some history of the Brain Preservation Prize, highlighting Ken Hayworth personally, and quotes a few experts saying we are nowhere close to being able to emulate brains. At one point she says,

The questions the couple faced may ultimately confront more of us with implications that could be preposterously profound.

Yet she discusses no such implications. She discusses no arguments on if emulation would be feasible or desirable or what implications it might have. I’ll give her the benefit of the doubt and presume that her priorities accurately reflect the priorities of New York Times readers. But those priorities are so different from mine as to highlight the question: what exactly do news readers want?

For a topic like this, it seems readers want colorful characters described in detail, and quotes from experts with related prestige. They don’t want to hear about arguments for or against the claims made, or to discuss further implications of those claims. It seems they will enjoy talking to others about the colorful characters described, and perhaps enjoy taking a position on the claims those characters make. But these aren’t the sort of topics where anyone expects to care about the quality or care of the arguments one might. It is enough to just have opinions.

Added 14Sep: Amy posted a related article that is a technical review of brain emulation tech. I’m glad it exists, but I also have nothing particular to say in response.

GD Star Rating
loading...
Tagged as: ,

The Ghosts Of Em

Our basic concept of “death” is binary, so that one is either dead or not. But we often metaphorically extend the concept to a continuum. For example, people who have more strength, energy, passion, and awareness are said to be “more alive,” and those who have more power, prestige, influence, or wealth are also said to be “more” in many ways, including more central and alive. Since sleepers have less of all of these things, sleep is often seen as a partial death.

We have a related mythical concept of “ghost,” which is also sometimes made into a continuum of ghostliness. A ghost was once human, but then died, and now is an active agent with death-related features. So ghosts tend to be cold, sick, in low mood, and have a weak influence on the physical world. They are typically distracted, unaware of, and disinterested in humans. Ghosts are anti-social, avoid groups of more than a few humans, and don’t collect into ghost gangs or ghost cities. They are reluctant to move away from their old haunts, and remain obsessed with old issues. Ghosts are heard more than seen, rarely speak words, and are seen more in unusual viewing modes such as night, shadows, and mirrors.

Slow em retirees share many features with people we see as “less alive,” including ghosts. Not only are they literally closer subjectively to dying soon due to civilization instability, their minds are also more inflexible and stuck in their ways. Compared to faster working ems, slow retirees have less awareness, wealth, status, and influence, and they are slower to respond to events, including via speaking words or coordinating with others. Retirees may often watch and judge working ems, and in such roles may only be visible only in special views.

Thus ems may come to see slower ems as ghostly, and more ghostly when slower. Such em ghosts are real, and with trouble one can talk to them, but like ghosts they aren’t very useful as allies, they sometimes hurt people they interact with, and so since one is usually free to ignore them, that is usually the wise strategy. Since ems must pay for faster speeds, for ems being more alive is more directly related to having more money to spend.

If “beneath” each em are many layers of a ghostly underworld, just how deep does this abyss go? Katja Grace at AI Impacts just helped me out by estimating the ratio of costs, using today’s technology, to store a brain state and to run a human-speed brain emulation. This ratio equals the “base” em speed as a fraction of human speed. This is near the lowest reasonable speed for ems, since well above it cost is proportional to speed, and well below it cost is independent of speed.

Apparently, plausible estimates of this base speed range from one hundredth of a trillionth of human speed up to one millionth of human speed, with a middle estimate of one tenth of a billionth of human speed. This ratio apparently hasn’t changed much over four decades, giving reason to hope it can help us estimate the future base speed. I’ve separately estimated typical em speed to be one thousand times human speed, and the maximum speed where speed is still proportional to cost to be a million times human speed.

Thus the range of speeds over which em speeds are about proportional to cost is at least a factor of a trillion, and may be a billion trillion. Thus for typical speed ems the underworld abyss of slower ghostly speeds is very deep! If your career and investments go badly, and you are forced to cut back and slow down, there is a very long hill to slide down before you finally reach bottom, where the only lower place to go is to be erased. Em inequality in speeds is immense.

I just added this stuff about ghostly ems to my book The Age of Em: Work, Love and Life When Robots Rule The Earth. And since I’m to turn in the final draft by Saturday, this will be the last thing I add. Publication date still not set; I’ll tell you when I know.

GD Star Rating
loading...
Tagged as: ,