Tag Archives: Ems

Oxford To Publish The Age Of Em

Eighteen months ago I asked here for readers to criticize my Em Econ book draft, then 62K words. (137 of you sent comments – thanks!) Today I announce that Oxford University Press will publish its descendant (now 212K words) in Spring 2016. Tentative title, summary, outline:

The Age Of Em: Work, Love and Life When Robots Rule The Earth

Author Robin Hanson takes an oft-mentioned disruptive future tech, brain emulations, and expertly analyzes its social consequences in unprecedented breadth and detail. His book is intended to prove: we can foresee our social future, not just by projecting trends, but also by analyzing the detailed social consequences of particular disruptive future technologies.

I. Basics
1. Start: Contents, Preface, Introduction, Summary
2. Modes: Precedents, Factors, Dreamtime, Limits
3. Mechanics: Emulations, Opacity, Hardware, Security
II. Physics
4. Scales: Time, Space, Reversing
5. Infrastructure: Climate, Cooling, Buildings
6. Existence: Virtuality, Views, Fakery, Copying, Darkness
7. Farewells: Fragility, Retirement, Death
III. Economics
8. Labor: Wages, Selection, Enough
9. Efficiency: Competition, Eliteness, Spurs, Power
10. Business: Institutions, Growth, Finance, Manufacturing
11. Lifecycle: Careers, Age, Preparation, Training
IV. Organization
12. Clumping: Cities, Speeds, Transport
13. Extremes: Software, Inequality, War
14. Groups: Clans, Nepotism, Firms, Teams
15. Conflict: Governance, Law, Innovation
V. Sociology
16. Connection: Mating, Signaling, Identity, Ritual
17. Collaboration: Conversation, Synchronization, Coalitions
18. Society: Profanity, Divisions, Culture, Stories
19. Minds: Humans, Unhumans, Intelligence, Psychology
VI. Implications
20. Variations: Trends, Alternatives, Transition, Aliens
21. Choices: Evaluation, Policy, Charity, Success
22. Finale: Critics, Conclusion, References, Thanks
23. Appendix: Motivation, Method, Biases

Added Sept2015: The book now has a website.

GD Star Rating
Tagged as: ,

Ian Morris on Foragers, Farmers, Industry, & Ems

The book Foragers, Farmers, and Fossil Fuels by Ian Morris will be published March 22. As I don’t see any other reviews on the web, it seems I get to be the first. This is from the publisher’s blurb:

Most people in the world today think democracy and gender equality are good, and that violence and wealth inequality are bad. But most people who lived during the 10,000 years before the nineteenth century thought just the opposite. … Fundamental long-term changes in values, Morris argues, are driven by the most basic force of all: energy. Humans have found three main ways to get the energy they need—from foraging, farming, and fossil fuels. Each energy source sets strict limits on what kinds of societies can succeed, and each kind of society rewards specific values. … The ongoing revolution in energy capture means that our most cherished values are very likely to turn out—at some point fairly soon—not to be useful any more.

I’m delighted that, like me, Morris divides human history into three great eras of foraging, farming, and industry. Furthermore, Morris suggests that a new era may start by 2082, perhaps based on brain emulations of the entire human population. He notes that these different past eras have been associated with dramatically different values, and suggests that the next era will also have very different values. So far remarkably similar to what I’ve been saying here for years!

Morris resists the idea that some eras have correct values while others have incorrect values. Instead he sees each era’s values as adapted to the environment of that era, i.e., to its technical methods of production and survival. Morris also sees the modes of energy production as central and even defining of those environments. Which is why he calls our industry era the “fossil fuel” era.

Morris does little to argue for the centrality of energy production tech in era environments. He doesn’t identify possible alternative centrality concepts with which to compare his view, nor does he offer evidence that might distinguish his energy-centrality from other views. Instead, Morris seems content to just assume energy centrality. While this stance didn’t at all persuade me of energy centrality, nothing anything else in his book seems to actually depend on this claim. So I’m happy to just set it aside, and focus on other issues.

Morris’s most interesting claim is that values during each era were adapted in great detail to the environments of those eras. And Morris fills up most of his book with details on both the environments and values of past eras. Enough details to make it clear that different eras did in fact have distinctively different environments and values. There are in fact typical forager environments, typical forager values, and so on for farming and industry. Yes there are exceptions, but that doesn’t invalidate the basic patterns.

However, Morris actually doesn’t try very hard to give specific explanations matching the specific features of each typical environment to specific features of each typical value set. It seems that his belief in strong adaptation of values to environments isn’t much based on such specific matches. Instead, Morris mainly just seems to be very impressed by how consistently different were the environments and values of each era. It is as if he reasons “why would all the farming values be like each other, and yet so different from foraging values, if not for being adaptations to the new distinct farming environment?”

Now I do pretty much accept this story regarding the foraging and farming eras. But this is because those eras lasted so very long, and we can see so much selection among units that could plausibly produce this adaptation. Foragers and farmers were both literally dirt poor, and so it didn’t take that much of a relative advantage to kill off one group and replace it with another. Foragers lasted for many thousands of generations, long enough to create enormous variance in the success of specific lineages and specific local cultures. And while the farming era lasted only a few hundred generations, we can see in history wave after wave after wave of cultures being displaced by other cultures, via war and famine and much else.

But while it is hard to deny great selection of cultures, including their values, during the foraging and farming eras, the case for selection seems to me to be far weaker for our industry era. Industry has seen less than a dozen generations of humans, and most of them are today rich enough to suffer little selection from insufficient material wealth. Yes, we have seen terrible wars, but they have been small and rare enough to impose only very mild selection pressures.

Now we do more plausibly see a lot of selection in industry era work and organization practices. Enough firms are born and die fast enough to accumulate a lot of selection pressure. In addition, to a modest degree firms can copy the practices at more successful firms, and so adapt without dying. And all this can plausibly explain a great many particular changes in the physical and social technologies used by such firms.

However, the “values” that Morris has in mind as being adapted to each era are grand things like favoring democracy, open markets, gender equality, and rule of law, and disfavoring violence, slavery, and wealth-inequality. The World Values Survey has tracked changes in such values and found that they are not much attributable to more successful nations displacing other nations, or even more successful people replacing others within a nation.

Instead the literature on cultural value change suggests that it is the same people who are changing their values over time, and that this change is caused to a substantial extent by increasing wealth. This does not look like selection at all, but looks instead like the revealing of a common internal conditionality in human values. Because our values are conditional on our wealth, they naturally move toward the industry-era value set as we get rich. My guess here is that we are reverting to forager era values, at least outside of work, as we less feel the strength of farmer-era pressures like fear, religion, and conformity.

In his book, Morris does mention that some people have challenged his claim that industry values are adapted by pointing to our low and falling industry era fertility rates, which seem very hard to understand as adaptive behavior. In response, Morris points out that we haven’t seen the long term effects of that low fertility yet, and notes that the low fertility rich might still win in the future by becoming highly copied brain emulations. But even if that ends up happening, it seems hard to see low industry fertility as an adaptation designed to produce that outcome.

But even if I disagree with Morris about the causes of industry era value changes, I can still agree that the values of the next era are likely to be quite different from industry era values, and that those values would be well adapted to that next era. While I’m not sure what reasons Morris would offer for that claim, my reasons are specific to my analysis of the details of a new era based on brain emulations.

In my analysis, wages fall to subsistence levels, margins of survival are slim, and competition is strong. That should plausibly reverse industry era changes due to increasing wealth per person, and create a lot of selection. In addition, greatly increased brain emulation speeds allow many generations of changes to happen in short clock times, allowing for more variation and selection of individuals and practices.

In sum, Morris gets an awful lot right about history, and about the future. I just wish he had attended a bit more to the details of how values get selected, and which values are in fact adaptive in which environments.

Added 9a: I gave no direct quotes because the book copy I have forbids that.

GD Star Rating
Tagged as: , , ,

Help Me Imagine 2

Back in April I asked readers to help me imagine:

[Your] community was … so successful … that one hundred exact copies of [it] were made then and spread around the world. They copied all the same people, work and play roles and relationships, even all the workspaces and homes. … Consider … your attitude toward the other copies of your group. On one hand, … you might want to have nothing to do with those other copies. … On the other hand, you might be eager to maximize your chances to share insights and learn from the other groups. (more)

Today let me ask for help imagining a different situation.

A copy of you was made when you were ten years old. That was a century ago. Since then many thousands of (exact) copies have been made from that one copy. Most of these copies have grown up to do one of the few jobs where there is a big demand for copies of you. (Copies of you have tried other jobs, but so far they’ve not been competitive.) Every year a few dozen more copies are made and trained for these same few careers, but using slightly newer methods, to adapt to changing customers, techs, etc. Older versions of you often help to train younger versions.

Now here are my key questions:

  1. As an older copy, how free would you feel to push advice on younger copies? You could advice them on work, friends, love, etc. When there were consequences for you, how strongly would you want to insist that they follow your advice?
  2. As a younger copy, how much would you trust the advice of older copies of yourself? How eager would you be to get and follow such advice, even when you didn’t understand it? How willing would you be to get into situations where you had to do what they told you?

Today as parents, teachers, mentors, etc. we often give advice to kids, students, and junior co-workers. But our eagerness to advise is tempered by knowing that they are often quite different from us, and in addition times can change, reducing the relevance of our earlier experience. Our eagerness to listen to such advice when young is also tempered by the same reasons. Even so, the two sides often find themselves in conflict, with older folks pushing more advice than the younger folks want to take.

When the advisor-advisee relation is between an older and younger copy of at the same person, instead of between two quite different people, is there more or less conflict in the relation? Is more or less advice given and taken to heart? Do people feel more or less free and autonomous?

Added 5p: Today if asked in the (far-view) abstract, people say they’d want to take advice from those more experienced than they. But they often feel different when they get a specific (near-view) piece of advice that they don’t want to follow, or when they feel a rivalry with the person giving the advice. Then people look for excuses not to follow the advice. I’d think the same processes would happen even with copies of yourself. So I ask you to imagine particular near-view situations, and not just to consider the question in far-view.

GD Star Rating
Tagged as: ,

“I Robot, You Unemployed”

Tomorrow (Wednesday) at 7pm EST I’ll do a Learn Liberty Live! web presentation on “I, Robot. You, Unemployed” here. After a short ten minute presentation, I’ll lead ninety minutes of discussion. I expect to focus on em econ.

GD Star Rating
Tagged as: ,

Em Software Results

After requesting your help, I should tell you what it added up to. The following is an excerpt from my book draft, illustrated by this diagram:


In our world, the cost of computing hardware has been falling rapidly for decades. This fall has forced most computer projects to be short term, so that products can be used before they are made obsolete. The increasing quantity of software purchased has also led to larger software projects, which involve more engineers. This has shifted the emphasis toward more communication and negotiation, and also more modularity and standardization in software styles.

The cost of hiring human software engineers has not fallen much in decades. The increasing divergence between the cost of engineers and the cost of hardware has also lead to a decreased emphasis on raw performance, and increased emphasis on tools and habits that can quickly generate correct if inefficient performance. This has led to an increased emphasis on modularity, abstraction, and on high-level operating systems and languages. High level tools insulate engineers more from the details of hardware, and from distracting tasks like type checking and garbage collection. As a result, software is less efficient and well-adapted to context, but more valuable overall. An increasing focus on niche products has also increased the emphasis on modularity and abstraction.

Em software engineers would be selected for very high productivity, and use the tools and styles preferred by the highest productivity engineers. There would be little interest in tools and methods specialized to be useful “for dummies.” Since em computers would tend to be more reversible and error-prone, em software would be more focused on those cases as well. Because the em economy would be larger, its software industry would be larger as well, supporting more specialization.

The transition to an em economy would greatly lower wages, thus inducing a big one-time shift back toward an emphasis on raw context-dependent performance, relative to abstraction and easier modifiability. The move away from niche products would add to this tendency, as would the ability to save copies of the engineer who just wrote the software, to help later with modifying it. On the other hand, a move toward larger software projects could favor more abstraction and modularity.

After the em transition, the cost of em hardware would fall at about the same speed as the cost of other computer hardware. Because of this, the tradeoff between performance and other considerations would change much less as the cost of hardware fell. This should greatly extend the useful lifetime of programming languages, tools, and habits matched to particular performance tradeoff choices.

After an initial period of large rapid gains, the software and hardware designs for implementing brain emulations would probably reach diminishing returns, after which there would only be minor improvements. In contrast, non-em software will probably improve about as fast as computer hardware improves, since algorithm gains in many areas of computer science have for many decades typically remained close to hardware gains. Thus after ems appear, em software engineering and other computer-based work would slowly get more tool-intensive, with a larger fraction of value added by tools. However, for non-computer-based tools (e.g., bulldozers) their intensity of use and the fraction of value added by such tools would probably fall, since those tools probably improve less quickly than would em hardware.

For over a decade now, the speed of fast computer processors has increased at a much lower rate than the cost of computer hardware has fallen. We expect this trend to continue long into the future. In contrast, the em hardware cost will fall with the cost of computer hardware overall, because the emulation of brains is a very parallel task. Thus ems would see an increasing sluggishness of software that has a large serial component, i.e., which requires many steps to be taken one after the other, relative to more parallel software. This sluggishness would directly reduce the value of such software, and also make such software harder to write.

Thus over time serial software will become less valuable, relative to ems and parallel software. Em software engineers would come to rely less on software tools with a big serial component, and would instead emphasize parallel software, and tools that support that emphasis. Tools like automated type checking and garbage collection would tend to be done in parallel, or not at all. And if it ends up being too hard to write parallel software, then the value of software more generally may be reduced relative to the value of having ems do tasks without software assistance.

For tasks where parallel software and tools suffice, and where the software doesn’t need to interact with slower physical systems, em software engineers could be productive even when sped up to the top cheap speed. This would often make it feasible to avoid the costs of coordinating across engineers, by having a single engineer spend an entire subjective career creating a large software system. For an example, an engineer that spent a subjective century at one million times human speed would be done in less than one objective hour. When such a short delay is acceptable, parallel software could be written by a single engineer taking a subjective lifetime.

When software can be written quickly via very fast software engineers, product development could happen quickly, even when very large sums were spent. While today investors may spend most of their time tracking current software development projects, those who invest in em software projects of this sort might spend most of their time deciding when is the right time to initiate such a project. A software development race, with more than one team trying to get to market first, would only happen if the same sharp event triggered more than one development effort.

A single software engineer working for a lifetime on a project could still have troubles remembering software that he or she wrote decades before. Because of this, shorter-term copies of this engineer might help him or her to be more productive. For example, short-term em copies might search for and repair bugs, and end or retire once they have explained their work to the main copy. Short-term copies could also search among many possible designs for a module, and end or retire after reporting on their best design choice, to be re-implemented by the main copy. In addition, longer-term copies could be created to specialize in whole subsystems, and younger copies could be revived to continue the project when older copies reached the end of their productive lifetime. These approaches should allow single em software engineers to create far larger and more coherent software systems within a subjective lifetime.

Fast software engineers who focus on taking a lifetime to build a large software project, perhaps with the help of copies of themselves, would likely develop more personal and elaborate software styles and tools, and rely less on tools and approaches that help them to coordinate with other engineers with differing styles and uncertain quality. Such lone fast engineers would require local caches of relevant software libraries. When in distantly separated locations, such caches could get out of synch. Local copies of library software authors, available to update their contributions, might help reduce this problem. Out of synch libraries would increase the tendency toward divergent personal software styles.

When different parts of a project require different skills, a lone software engineer might have different young copies trained with different skills. Similarly, young copies could be trained in the subject areas where some software is to be applied, so that they can better understand what variations will have value there.

However, when a project requires different skills and expertise that is best matched to different temperaments and minds, then it may be worth paying extra costs of communication to allow different ems to work together on a project. In this case, such engineers would likely promote communication via more abstraction, modularity, and higher level languages and module interfaces. Such approaches also become more attractive when outsiders must test and validate software, to certify its appropriateness to customers. Enormous software systems could be created with modest sized teams working at the top cheap speed, with the assistance of many spurs. There may not be much need for even larger software teams.

The competition for higher status among ems would tend to encourage faster speeds than would otherwise be efficient. This tendency of fast ems to be high status would tend to raise the status of software engineers.

GD Star Rating
Tagged as: , ,

Em Software Engineering Bleg

Many software engineers read this blog, and I’d love to include a section on software engineering in my book on ems. But as my software engineering expertise is limited, I ask you, dear software engineer readers, for help.

“Ems” are future brain emulations. I’m writing a book on em social implications. Ems would substitute for human workers, and once ems were common ems would do almost all work, including software engineering. What I seek are reasonable guesses on the tools and patterns of work of em software engineers – how their tools and work patterns would differ from those today, and how those would vary with time and along some key dimensions.

Here are some reasonable premises to work from:

  1. Software would be a bigger part of the economy, and a bigger industry overall. So it could support more specialization and pay more fixed costs.
  2. Progress would have been made in the design of tools, languages, hardware, etc. But there’d still be far to go to automate all tasks; more income would still go to rent ems than to rent other software.
  3. After an initial transition where em wages fall greatly relative to human wages, em hardware costs would thereafter fall about as fast as non-em computer hardware costs. So the relative cost to rent ems and other computer hardware would stay about the same over time. This is in stark contrast to today when hardware costs fall fast relative to human wages.
  4. Hardware speed will not rise as fast as hardware costs fall. Thus the cost advantage of parallel software would continue to rise.
  5. Emulating brains is a much more parallel task than are most software tasks today.
  6. Ems would typically run about a thousand times human mind speed, but would vary over a wide range of speeds. Ems in software product development races would run much faster.
  7. It would be possible to save a copy of an em engineer who just wrote some software, a copy available to answer questions about it, or to modify it.
  8. Em software engineers could sketch out a software design, and then split into many temporary copies who each work on a different part of the design, and talk with each other to negotiate boundary issues. (I don’t assume one could merge the copies afterward.)
  9. Most ems are crammed into a few dense cities. Toward em city centers, computing hardware is more expensive, and maximum hardware speeds are lower. Away from city centers, there are longer communication delays.

Again, the key question is: how would em software tools and work patterns differ from today’s, and how would they vary with time, application, software engineer speed, and city location?

To give you an idea of the kind of conclusions one might be tempted to draw, here are some recent suggestions of François-René Rideau: Continue reading "Em Software Engineering Bleg" »

GD Star Rating
Tagged as: ,

I Still Don’t Get Foom

Back in 2008 my ex-co-blogger Eliezer Yudkowsky and I discussed his “AI foom” concept, a discussion that we recently spun off into a book. I’ve heard for a while that Nick Bostrom was working on a book elaborating related ideas, and this week his Superintelligence was finally available to me to read, via Kindle. I’ve read it now, along with a few dozen reviews I’ve found online. Alas, only the two reviews on GoodReads even mention the big problem I have with one of his main premises, the same problem I’ve had with Yudkowsky’s views. Bostrom hardly mentions the issue in his 300 pages (he’s focused on control issues).

All of which makes it look like I’m the one with the problem; everyone else gets it. Even so, I’m gonna try to explain my problem again, in the hope that someone can explain where I’m going wrong. Here goes.

“Intelligence” just means an ability to do mental/calculation tasks, averaged over many tasks. I’ve always found it plausible that machines will continue to do more kinds of mental tasks better, and eventually be better at pretty much all of them. But what I’ve found it hard to accept is a “local explosion.” This is where a single machine, built by a single project using only a tiny fraction of world resources, goes in a short time (e.g., weeks) from being so weak that it is usually beat by a single human with the usual tools, to so powerful that it easily takes over the entire world. Yes, smarter machines may greatly increase overall economic growth rates, and yes such growth may be uneven. But this degree of unevenness seems implausibly extreme. Let me explain. Continue reading "I Still Don’t Get Foom" »

GD Star Rating
Tagged as: , , ,

Paul Carr Interviews Me

In this episode of the Wow! Signal Podcast. The topic is ems, starting about minute 35, after an interview with Heath Rezabek.

GD Star Rating
Tagged as: ,

First Person Em Shooter

Jesse Galef:

It’s The Matrix meets Braid: a first-person shooter video game “where the time moves only when you move.” You can stare at the bullets streaking toward you as long as you like, but moving to dodge them causes the enemies and bullets to move forward in time as well. The game is called SUPERHOT … it struck me: this might be close to the experience of an emulated brain housed in a regular-sized body.

Jesse asked for my reaction. I said:

Even better would be to let the gamer change the rate at which game-time seems to move, to have a limited gamer-time budget to spend, and to give other non-human game characters a similar ability.

Jesse riffed:

It would be more consistent to add a “mental cycle” budget that ran down at a constant rate from the gamer’s external point of view. I don’t know about you, but I would buy that game! (Even if a multi-player mode would be impossible.)

Let’s consider this in more detail. There’d be two plausible scenarios:

Brain-In-Body Shooter – The em brain stays in a body. Here changing brain speeds would be accomplished by running the same processors faster or slower. In this case, assuming reversible computing hardware, the em brain computing cost for each subjective second would be linear in brain speed; the slower the world around you moved, the more you’d pay per gamer second. This would be an energy cost, to come out of the same energy budget you used to move your body, fire weapons, etc. There would also probably be a heat budget – you’d have some constant rate at which cooling fluids flow to remove heat, and the faster your mind ran the faster heat would accumulate to raise your temperature, and there’d be some limit to the temperature your hardware would tolerate. Being hot might make your body more visible to opponents. It would hard for a video game to model the fact that if your body is destroyed, you don’t remember what happened since your last backup.

Brain-At-Server Shooter – The em brain runs on a server and tele-operates a body. Here switching brain speeds would usually be accomplished by moving the brain to run on more or fewer processors at the server. In this case, em brain computing cost would be directly proportional to subjective seconds, though there may be a switching cost to pay each time you changed mental speeds. This cost would come out of a financial budget of money to pay the server. One might also perhaps allow server processors to temporarily speed up or slow down as with the brain-in-body shooter. There’d be a serious risk of opponents breaking one’s net connection between body and brain, but when your body is destroyed at least you’d remember everything up to that point.

To be able to switch back and forth between these modes, you’d need a very high bandwidth connection and time enough to use it lots, perhaps accomplished at a limited number of “hard line” connection points.

Not that I think shooter situations would be common in an em world. But If you want to make a realistic em shooter, these would be how.

GD Star Rating
Tagged as: ,

Em Econ @ Yale Thursday

The Yale Technology & Ethics study group hosts about one talk a month on various futurist topics. Amazingly, I was their very first speaker when the group started in 2002. And this Thursday I’ll return to talk on the same subject:

The Age of Em: Social Implications of Brain Emulations

4:15-6:15pm, May 22, Yale ISPS, 77 Prospect St (corner of Prospect & Trumbull), Rm A002.

The three most disruptive transitions in history were the introduction of humans, farming, and industry. If another transition lies ahead, a good guess for its source is artificial intelligence in the form of whole brain emulations, or “ems,” sometime in the next century. I attempt a broad synthesis of standard academic consensus, including in business and social science, in order to outline a baseline scenario set modestly far into a post-em-transition world. I consider computer architecture, energy use, cooling infrastructure, mind speeds, body sizes, security strategies, virtual reality conventions, labor market organization, management focus, job training, career paths, wage competition, identity, retirement, life cycles, reproduction, mating, conversation habits, wealth inequality, city sizes, growth rates, coalition politics, governance, law, and war.

My ’02 talk was controversial; Thursday’s talk will likely be well. All are welcome.

Added 28May: Audio, slides.

GD Star Rating
Tagged as: , ,