Tag Archives: Ems

Old Minds Are Fragile

Consider three design problems:

  • Case #1: First, you are asked to modify a stock car, making it into a truck to haul stuff. After you do that, you are asked to create a race car. Which would you rather start from for this second task, another stock car, or the stock car that you turned into a truck?
  • Case #2: A species of beetle lives in a varied and changing environment, and so has a rather simple and basic design. Some of these beetles invade a different and more stable environment, and acquire adaptations specific to that environment. A third rather different but also stable environment opens up adjacent to both previous environments. Which beetle type’s descendants will likely fill this third environment?
  • Case #3: Over the last decade a group wrote software to do a certain task (e.g., print driver, web server, spreadsheet, etc.) This design of this software was matched to certain features of the problem environment, such as hardware, network speeds, etc. Today there is a need for software to do a similar task, except that the problem environment has changed. To write this new software, would you have your team modify this previous software, or start a new system mostly from scratch?

In all these cases, one makes a system to function in a given environment, and can either modify a complex system adapted to a different environment, or ”start over” via modifying a simpler system less adapted to any specific environment. In general, the more different is the new environment from the old, the better it is to start over. Old  systems tend to be rigid, which makes them fragile, in that they break if you bend them too far.

This suggests that designed systems tend to get irreversibly fragile as they adapt to specific environments. When context changes greatly, it is usually easier to build new systems from “scratch,” than to un-adapt systems designed for other contexts. Software tends to “rot“, for example.

An empirical prediction here is that species occupying highly variable environments tend to have more descendant species in other environments, compared to species occupying less variable environments. I don’t know if this has been tested. It fits with the Innovator’s Dilemma though, where firms who serve the low end of a product line with simpler techs tend to creep up and displace those serving the high end; high end products tend to be more complex.

Today I’m focused on this being bad news for the feasibility of immortality, at least for human-like creatures. You see, our minds seem designed to adapt to the environment in which we grow up, via youthful plasticity transitioning to elderly rigidity. For example, we are great at learning languages when young, and terrible when old. We are similarly receptive when young to new ways to categorize and conceive of things, but once we have often used particular ways, we find it harder to understand and use alternatives.

The brains of most animals peak in functionality during their key reproductive years, and do worse both before and after. Short lived animals peak sooner than long lived animals. Some of the early rise is due to learning, and some of later decline is due to the decline of individual cells and connections. Some of this pattern may even be due to an explicit plan to turn up some dials on plasticity early on, and then turn down those dials later. But I think another important part of this rise and fall is due to a general robust tendency for adapted systems to slide from plasticity to rigidity.

Thus even if we succeed in creating emulations of whole human brains, “ems” which can use backups, body swaps, etc. to avoid bodily death and decay, we should expect such ems to decay by getting mentally rigid with subjective age. Even if we do not emulate any decline in individual cell and connection performance, nor any age-specific general plasticity dial settings, the mind itself may well decay with subjective experience, because such decay is just intrinsic to mind design.

Now in software design one can often slow a slide to rigidity by refactoring code, such as by looking for better abstractions to achieve modularity. But the brain probably already has some analogues to refractoring, such as in its ways to reorganize concepts. And even with large refactoring efforts, most designed software eventually gets rigid, so that when environments change enough such software is replaced wholesale by new systems built from scratch.

Similarly, em workers who start out subjectively young, and then learn how to work in a stable environment, may become increasingly productive in that environment, even after thousands of years of subjective experience. But when a new quite different work environment appears, one can probably gain more work productivity by training subjectively young ems for it, rather than trying to change ems who had spend thousands of subjective years adapting to a very different environment.

Today most houses and cars are in principle immortal, in the sense that enough maintenance can keep them functioning indefinitely. Yet most houses and cars are not immortal in practice, because those maintenance costs keep rising to the point where it is cheaper to build new houses and cars. Similarly it might be possible to keep very old ems around, even when they have become much less productive because relevant environments have changed. Someone, however, would have to pay that cost, relative to the option of using more productive younger ems. And as with houses and cars today, maybe few will pay.

If you personally hope to become an em with an especially long productive subjective life, it is probably important to stay general and flexible for as long as you can. Prefer to acquire habits and insights that are widely applicable, and whose value is likely to long continue. Prefer to write, deal with people, and manage complexity, rather than learning the detailed layout of a city or how best to write in a particular new programming language.

Eventually we may find mind designs with a much weaker tendency toward rigidity with age. And we may find ways to transfer some important elements of once-human minds, such as their memory and personality, into this alternative framework. But even then there should be some aging. And it gets even less clear if you’d want to think of such a changed creature as you.

Even more eventually, the universe should get a lot more stable, and with it the environments where minds function. Then there will be a lot more scope for very long lived human-like minds. If there are any human-like minds left at that point.

Added: Stem cells fit this; bodies usually make cells designed for specific places from general simpler stem cells, not by changing other specific cells.

GD Star Rating
loading...
Tagged as: , ,

Work, Play Extremes

Humanity’s mix of work vs. play has varied over the millennia. Farmers played less less than foragers, and with industry we’ve moved back to, and even past, forager levels of play. In the future, the work-play mix could move in either direction, possibly to extremes.

Pause for a moment to ask yourself: which extreme do you most fear, a mostly-work future, or a mostly-play future? Yes, all else equal play is probably better than work, but all else may not be equal – ask yourself what knowing that a world is mostly-work or mostly-play would tell you about the rest of that world.

Me, I more fear a mostly-play future. I fear a world of people so overwhelmed by the pleasures of music, movies, games, virtual reality, drugs, etc. that they don’t build for the future, or even maintain support structures inherited from the past. Failing to invest in capital or children, humanity shrinks and falls into oblivion.

Yes, there are things to fear about a mostly-work future. Mainly, the opportunity cost of fun not had. Play is often more fun, and even fulfilling, than work. Given a momentary choice, we tend to choose play over work, and for good reason. Even so, I’d expect a mostly-work world to continue to invest and grow, building to a larger population and capacity. So that if later that world devolves into most-play, at least more people will have more fun on the way down.

Notice that this issue suggests that status isn’t such a bad thing. Locally, the possibility of efforts to gain status seem to cause a market failure, as your status gains come at the expense of the status of others. This would seem to make us work too hard to gain status. But since we can more reliably gain status via work than via play, the existence of status pushes us toward work, and away from the more dangerous mostly-play extreme.

I’m not thrilled that the em future I envision is a mostly-work world. But it at least seems safer than the other extreme to which the work-play mix could have evolved.

GD Star Rating
loading...
Tagged as: , , ,

I Talk At Oxford Wednesday

I’ll speak at Oxford this Wednesday, 4:00pm, on:

Em Econ 101

The three most disruptive transitions in our history coincided with the introduction of humans, farming, and industry. If another such transition lies ahead, a good guess for its source is artificial intelligence in the form of whole brain emulations, or “ems.” Most who consider ems discuss their implications for the philosophy of identity, or their feasibility and development paths. Those who consider em social implications gravitate toward heaven or hell scenarios, or invent entirely new economics, etc. for this new era.

In contrast, as a professor of economics I seek to straight-forwardly apply standard economic and other social science theory to these novel technical assumptions, to sketch rough outlines of a relatively-likely reference scenario set modestly far into a post-em-transition world. I consider how ems might change: reproduction, life plans, cycles of daily life, inequality, work training, property rights, families, firm management, industrial organization, urban agglomeration, security, and governance.

Location: Oxford Martin School, 34 Broad Street, OX1 3BG

Added 29June: Here are talk slides, audio.

Added 2July: Here is Video.

GD Star Rating
loading...
Tagged as: ,

Forager, Farmer Morals

Looking for insight into farmer-era world views, I just read the 1931 novel The Good Earth, about Chinese farmers. It is of course more a morality tale than a documentary, and the main character soon gets rich, and is then no longer a representative farmer. But the story illustrates differences between farmer vs. forager style morality.

Foragers live in close egalitarian bands, with behavior well adapted to their environment. So forager morality issues are mostly about well-adapted personal behavior in conflict with group interests. Foragers sin by bragging, not sharing, being violent against associates, etc.

Farmer morality, in contrast, is much more about conflicts within people than within groups. Farmers sin by being lazy, wanting overly fancy foods, taking drugs, having sex with prostitutes, wanting status markers that cost too much in the long run, etc. Farmers need to resist internal temptations to do things that might make sense for foragers, but which can ruin farmers. These can also ruin one’s family and friends, so farmer sins also have shades of selfishness.

Of course farmers also care about bragging, violence, etc. In some sense farmers have more morality – more and stronger rules, to fight against stronger natural inclinations. So farming culture introduced religion and stronger social pressures to enforce their rules, to keep farmers from relapsing into foragers.

This helps me make sense of Jonathan Haight’s observations that liberals, who I’ve called forager-like, rely on fewer moral principles than conservatives, who I’ve called farmer-like:

The current American culture war, we have found, can be seen as arising from the fact that liberals try to create a morality relying primarily on the Care/harm foundation, with additional support from the Fairness/cheating and Liberty/oppression foundations. Conservatives, especially religious conservatives, use all six foundations, including Loyatly/betrayal, Authority/subversion, and Sanctity/degradation. (more)

I’ve suggested that as we’ve become richer, we’ve become more forager-like. If our descendants get poor again, they’ll probably need stronger social norms again, to get them to resist temptations to act like foragers and do what is functional in their world. Their morality would probably rely on a wider more-conservative-like range of moral feelings.

In the em scenario I’ve been discussing here, sex would be unimportant except as a possible way to waste too much time. So em morality would be pretty liberal on sex. But money, work, and reputation would be important – ems would probably have pretty conservative attitudes on keeping their word, doing their job, obeying their boss, and not stealing. When mind theft or virus corruption are big risks, they’d also probably have strong purity feelings about avoiding acts that could risk such harms. And they’d probably feel strong clan loyalty, even beyond what farmers feel, to the clan of copies of the same original human.

GD Star Rating
loading...
Tagged as: , , ,

Those Who Love Work

Christina Alger expresses her, and her dad’s, passion for work:

I was 6 at the time, maybe 7 … I was playing Office. In order to play Office, I had to get into character. I would don one of my dad’s suit jackets — I preferred a nice gray pinstripe — and would attempt to balance a spare pair of his glasses on my small snub nose. Sometimes I would shuffle around in his wingtips. Then I would organize piles of papers on my desk, filing them away in folders once they had been properly reviewed. …

My dad’s office … felt like Cheers: it was a place where I could relax after a long day, where everyone knew my name. … Dad was always willing to give me tasks that made me feel important. … My dad was then, and remains to this day, one of the few grown-ups I have come across who truly loved his job. … His enthusiasm for work was infectious. Dad loved playing office; why wouldn’t I? …

“Isn’t writing from home lonely?” My friend Anne asked me over coffee. “I have this vision of you stuck in your apartment all day, talking to imaginary people.” … “I’m an only child,” I shrugged. “I like being alone.” … Dad would like my new office, I think. He would see how much I enjoy working here. Whenever I have a successful day of writing, I wish I could share it with him. But not once have I ever felt lonely. (more)

Sure some who pretend to love work are fooling themselves, but I’m pretty sure that many others like Christina do honestly love their work. Yes Christina and her dad probably enjoy their work more because it is high status, but it is hard to believe their work love is entirely status driven – they’d probably still love their work, if a bit less, if it were low status.

It seems likely that we could find a few hundred humans like Jiro, Christina and her dad, productive folks who love work even when they put in many hours, and who are willing to adapt to the changed world ems would inhabit. If so, that world could be a vast wonderful world full of life and fulfillment.

GD Star Rating
loading...
Tagged as: ,

Jiro Lives Worth Living

Jiro Dreams of Sushi is a well-reviewed documentary mostly celebrating the world’s best sushi chef. Which is worth pondering, because Jiro is an extreme workaholic. Roger Ebert:

Jiro Ono is 85 years old. As a young boy, he ran away from home to become an apprentice in a restaurant and has been making sushi for more than 70 years. He is apparently not happy doing anything else and prefers to work all day, seven days a week, every day in the year. … You realize he must be a rich man. But to what end? … While watching it, I found myself drawn into the mystery of this man. Are there any unrealized wishes in his life? Secret diversions? Regrets? If you find an occupation you love and spend your entire life working at it, is that enough? (more)

Jiro’s life is also quite routine – he repeats the same actions over and over and over, always seeking slight adjustments to improve quality.

I haven’t seen anyone says this movie describes a terrible tragedy of a wasted worthless life. Most seem to accept Jiro’s life as worth living, and many consider his an exemplary life. Yet let’s imagine some variations on Jiro’s life, and ask if they are also worth living.

First, imagine that Jiro is not rich. He is still the very best, but he gives his sushi away. He has enough to eat, stays warm, and is healthy, but has few luxuries. But since he spends most of his time at the office, it probably doesn’t make that much difference to his quality of life if he is rich or poor.

Second, imagine someone with Jiro’s unsurpassed skill, overwhelming dedication, and fascination with their work, except that this person makes plywood, not sushi. Would that also be a life worth living? It would be a lower status life, as our culture lauds sushi chefs more than plywood makers. But he would still be the very best plywood maker in the world. Isn’t that enough?

Third, imagine holding constant this person’s skill, while increasing other workers’ skills, so that this person is now only of median quality. His subjective experience of working on the job would be similar, except he couldn’t feel superior to everyone else. Would his life be worth living then? That is, can status by itself make the difference between a life worth living and one not? If when he isn’t noticing his status, he has the same feeling of flow, immersion, and fascination in his work, wouldn’t that be enough for a life worth living?

Some of you probably see where I am going with this. Imagine we take the few hundred very best most dedicated workaholic humans, and fill a world with trillions of em copies of them, so that they are mostly working at near subsistence wages, yet have enough food, warmth, health, etc. Is this a world full of creatures with lives worth living?

GD Star Rating
loading...
Tagged as: , ,

Leigh on Ems

Economist turned politician Andrew Leigh writes on “Five science breakthroughs that will transform politics“:

In this article, I’ve focused on ideas that are just over the horizon for most of us. So green roofs, LED lights, genetically modified crops, 3D printers and geo-engineering are important, but improvements are likely to be steady rather than seismic. Instead, I’ve chosen “disruptive ideas” that could radically affect the way our society operates. … 1. Driverless electric cars … 2. Space elevators … 3. Nanotechnology … 4. Ubiquitous Data … 5. Machine Intelligence …

He has many thoughtful policy comments on the first four topics, but when he gets to machine intelligence, he throws up his hands:

A machine that can emulate the human brain would challenge all occupations, from hairdressers to architects. In the case of this science breakthrough, it’s hard to even begin to think how policymakers would respond. Do we limit how many times you can replicate yourself? If we have a machine that contains your memories and can think like you, shall we treat it like a slave or pay it a wage? Do you have the right to turn off copies of yourself? Will this breakthrough cause wages to fall? If so, how do we make sure that everyone has some capital to get by? After thinking about Hanson’s work for a few weeks, I’ve decided that this is one breakthrough for which I don’t want to be around.

Leigh’s attitude makes sense to me. After spending years getting expert in thinking about good policy for this our industrial era, Leigh can see that ems are a whole new era where policy must be re-thought, starting back from basics. He doesn’t want to do that – he’d rather build on the expertise he has acquired to attack our many important industrial era problems. I hope he succeeds at that.

I also hope that when people like me do think through em policy more carefully, people with Leigh’s good sense and connections will listen.

GD Star Rating
loading...
Tagged as:

Anissimov On Ems

In an article titled “What are the Benefits of Mind Uploading?” Michael Anissimov lists seven benefits:

If the early adopters don’t go crazy and/or use their newfound abilities to turn the world into a totalitarian dictatorship, … others will then follow. … Suppose that millions of people choose to go for it. Widespread uploading would have huge effects. …

  1. Growth rates in human capital of 1,000% per year or far more. …
  2. Many of the details of human cognition would be elucidated and could be enhanced. …
  3. Reprogram their own brains to raise their happiness set points. …
  4. Consume far less space and use less energy and natural resources than we would in a conventional human body. … Avoid all the environmental destruction caused by clear-cutting land for farming. …
  5. A personal virtual sandbox could become one’s canvas for creating the fantasy world of their choice. …
  6. By offering partial readouts of our cognitive state to others, we could engage in a deeper exchange of ideas and emotions. …
  7. Last but not least, indefinite lifespans. …

The number of new minds leading worthwhile lives that could be created using the technology would be astronomical.

So why does Anissimov write an article only on uploading’s upsides? He doesn’t say he’ll soon post a companion article on downsides. So can he not think of downsides? Or does he see his readers as only interested in upsides? What kind of readers would want an article only on the upsides of something anyway?

Sadly, this makes Anissimov seem like he’s selling something, to fans who want to be sold. And alas many do seem to have a core belief that the future will be great, and a zeal to read articles by like-minded folks.

My approach, I hope it is clear, is not to sugar-coat the many downsides of em/uploads, or any other aspect of the future, even if I expect gains overall. So let’s list what many would see as em downsides, matched to Anissimov’s upsides:

  1. With faster growth, older generations overlap more with new generations. Humans can more quickly lose their importance and influence, and still be alive to see descendants reject things they hold dear.
  2. Em cognition might be changed to emphasize work over leisure capacities.
  3. Em cognition might also be changed to take more happiness from work, and to accept more inequality and workplace domination.
  4. An astronomical number of new minds may take more total resources than humans do now, and take less care to protect nature, as nature’s death won’t threaten their death as it does for us.
  5. Ems may spend less time in leisure than we, and less in fantasy VR than we do TV and video games.
  6. Employers and police may use direct access to cognitive states to test effort and loyalty, and to enforce rules.
  7. Only a small minority may be able to afford indefinite lifespans. Many em lives might be very short restarts from a standard trained start.
GD Star Rating
loading...
Tagged as:

Em Need For Speed

I recently found fault with Keith Henson’s assumption that sexual competition would induce ems to run as fast as physically possible. So how fast do I think ems would run? Here is my current analysis:

Em speeds should intersect supply and demand. Speed supply comes from how em hardware (e.g., device, energy, and cooling) costs vary with speed. Since human brains use a very parallel design with cells whose signals change far slower than electronic circuits, the cost of em hardware should be roughly linear in em speed across a wide range, to a very fast max, perhaps a million times faster than humans. In this range, thinking twice as fast costs about twice as much.

Above that linear regime, a 1% speedup will add more than 1% to costs, with this speed premium approaching infinity at a maximum feasible speedup, perhaps a factor of a billion. Very slow ems should also suffer a cost premium, as they’d still need to store a mental state.

With compatible hardware, brief speed increases might be cheap if em brains have substantial heat capacity. Longer but still temporary speed changes might be made by swapping into different brain hardware, though this could have substantial switching costs.

On the demand for em speed, I see seven relevant factors:

  1. When physical systems have natural resonance periods, managing those systems suggets em response times near the shortest of those periods. For example, since small moveable human body parts have resonance periods of a fraction of a second, human brains have reaction times on that time scale – reacting faster might help sometimes, but costs too much. Ems with smaller human-like bodies would want faster brains to match their shorter periods.
  2. Ems that talk often would benefit from having similar mind speeds. This would create a tendency for em speeds to clump at common standard speeds. Ems that talk often to humans would have near human speeds. Ems with highly mismatched speeds could talk naturally if the slow one temporarily moved to faster mental hardware.
  3. It is awkward for ems to talk when there are substantial communication delays. For any given distance to em conversation partners, there is some max speed above which delays are noticeable and hence costly.
  4. It is tempting to use faster ems to speed up any project whose duration might take a substantial fraction of the economy’s doubling time, or where there is a race with competing projects. Of course project durations may be limited by factors other than em thinking speeds.
  5. The more important is a negotiation or argument between ems, the more private gains can come from having a faster em mind, to out-think the other ems. So in hierarchical organizations, higher level leaders would have faster minds.
  6. When it is useful to coordinate two different tasks, one could either have two ems do the two tasks and talk periodically, or have a single faster em do both tasks. A single em doing both tasks probably has skills less well matched to those tasks, and would pay extra costs to switch between tasks. But when task coordination is important enough, these can be prices worth paying.
  7. When it is important to minimize the time a worker is away from their tasks at leisure and sleep, it will be tempting to run those non-work activities very fast. This could allow near continuous time coverage of a task.

Thus while some ems will have speeds to match the physical systems they manage, and ems would be faster at sleep, leisure, on thinking-dominated projects, and at high organization levels. The speed of other ems would be set more by how important is coordination for their tasks, and em speeds would tend to clump.

Coordination seems especially important in key design tasks, and in management. For example, it would be especially tempting to have all the parts of a large intricate software project written by the same very fast em. It would also be tempting to have the top thousand or more manager roles in a big organizations all be filled by a single very fast em.

Faster ems would naturally tend to be richer ems, if nothing else because they’d have some discretion in how they used their time, and that time is worth more. Thus a single very fast boss could afford to own more of a firm, reducing owner vs. manager conflicts.

If faster ems tend to be richer, win arguments, and fill key design and management roles, they would naturally be treated as higher status, at least by our status cues. Ems would also likely see them as higher status.

Social roles can often be usefully divided into roles that deal more with insiders, vs. roles that deal more with outsiders. For example, in a family, childcare is an inside role, while working for money is an outside role. In a hierarchical organization, managers have a more outside role – they deal more with outsiders. We care more about openness and helpfulness in inside roles, but more about opacity and toughness in outside roles.

When ems of different speeds meet, the slower em would naturally be more transparent and the faster one more opaque. It seems that faster ems would tend more to take on outside roles, which will be associated with higher status. In hierarchical organizations, subordinates might be expected to be open, such as via allowing direct hardware access to their emotional expressions, while bosses might typically hide their feelings from subordinates.

The overall picture here seems to be of even more inequality than I had imagined when I just considered wealth inequality among a larger future population whose lifespans vary more. Each em firm may have one very fast rich dominant boss who personally owns a lot of the firm. All front line managers might report to this one super boss, in meetings where they temporarily run at boss speeds, and are expected to be emotionally open to boss inspection. Sir, yes sir!

All else equal, an increase in the spatial extent of a firm or city would tend to reduce the speed of ems that might notice substantial communication delays. If em firms and cities tend to naturally grow larger over time, they’d also tend to naturally become slower, at least at their peak speeds. The gains that the would have otherwise achieved from faster speeds would be compensated by being able to interact naturally with a wider range of ems.

GD Star Rating
loading...
Tagged as: , , , ,

Henson On Ems

Keith Henson, of whom I’ve long been a fan, has a new article where he imagines our descendants as fragmenting Roman-Empire-like into distinct cultures, each ~300 meter spheres holding ~30 million ems each ~1 million times faster than a human, using ~1TW of power, in the ocean for cooling. The 300m radius comes from a max two subjective seconds of communication delay, and the 30 million number comes from assuming a shell of ~10cm cubes, each an em. (Quotes below)

The 10cm size could be way off, but the rest is reasonable, at least given Henson’s key assumptions that 1) competition to seem sexy would push ems to run as fast as feasible, and 2) the scale of em “population centers” and culture is set by the distance at which talk suffers a two subjective seconds delay.

Alas those are pretty unreasonable assumptions. Ems don’t reproduce via sex, and would be selected for not devoting lots of energy to sex. Yes, sex is buried deep in us, so ems would still devote some energy to it. But not so much as to make sex the overwhelming factor that sets em speeds. Not given em econ competitive pressures and the huge selection factors possible. I’m sure it is sexy today to spend money like a billionaire, but most people don’t because they can’t afford to. Since running a million times faster should cost a million times more, ems might not be able to afford that either.

Also, the scale at which we can talk without delay has just not been that important historically in setting our city and culture scales. We had integrated cultures even when talking suffered weeks of delay, we now have many cultures even though we can all talk without much delay, and city scales have been set more by how far we can commute in an hour than by communication delays. So while ems might well have a unit of organization corresponding to their easy-talk scale, important interactions should also exist at larger scales.

Those promised quotes from Henson’s article: Continue reading "Henson On Ems" »

GD Star Rating
loading...
Tagged as: , ,