Tag Archives: Ems

Em- vs Non-Em- AGI Bet

Joshua Fox and I have agreed to a bet:

We, Robin Hanson and Joshua Fox, agree to bet on which kind of artificial general intelligence (AGI) will dominate first, once some kind of AGI dominates humans. If the AGI are closely based on or derived from emulations of human brains, Robin wins, otherwise Joshua wins. To be precise, we focus on the first point in time when more computing power (gate-operations-per-second) is (routinely, typically) controlled relatively-directly by non-biological human-level-or-higher general intelligence than by ordinary biological humans. (Human brains have gate-operation equivalents.)

If at that time more of that computing power is controlled by emulation-based AGI, Joshua owes Robin whatever $3000 invested today in S&P500-like funds is worth then. If more is controlled by AGI not closely based on emulations, Robin owes Joshua that amount. The bet is void if the terms of this bet make little sense then, such as if it becomes too hard to say if capable non-biological intelligence is general or human-level, if AGI is emulation-based, what devices contain computing power, or what devices control what other devices. But we intend to tolerate modest levels of ambiguity in such things.

[Added 16Aug:] To judge if “AGI are closely based on or derived from emulations of human brains,” judge which end of the following spectrum is closer to the actual outcome. The two ends are 1) an emulation of the specific cell connections in a particular human brain, and 2) general algorithms of the sort that typically appear in AI journals today.

We bet at even odds, but of course the main benefit of having more folks bet on such things is to discover market odds to match the willingness to bet on the two sides. Toward that end, who else will declare a willingness to take a side of this bet? At what odds and amount?

My reasoning is based mainly on the huge costs to create new complex adapted systems from scratch when existing systems embody great intricately-coordinated and adapted detail. In such cases there are huge gains to instead adapting existing systems, or to creating new frameworks to allow the transfer of most detail from old systems.

Consider, for example, complex adapted systems like bacteria, cities, languages, and legal codes. The more that such systems have accumulated detailed adaptations to the detail of other complex systems and environments, the less it makes sense to redesign them from scratch. The human mind is one of the most complex and intricately adapted systems we know, and our rich and powerful world economy is adapted in great detail to many details of those human minds. I thus expect a strong competitive advantage from new mind systems which can inherit most of that detail wholesale, instead of forcing the wholesale reinvention of substitutes.

Added 16Aug: Note that Joshua and I have agreed on a clarifying paragraph.

GD Star Rating
loading...
Tagged as: , , ,

Me On Big Picture Science

I’m the first of four folks interviewed for the “Meet Your Replacements” episode of the SETI Institute’s Big Picture Science radio show. The topic:

There’s no one like you. At least, not yet. But in some visions of the future, androids can do just about everything, computers will hook directly into your brain, and genetic human-hybrids with exotic traits will be walking the streets. So could humans become an endangered species?

Be prepared to meet the new-and-improved you. But how much human would actually remain in the humanoids of the future?

Plus, tips for preventing our own extinction in the face of inevitable natural catastrophes.

My part is with Seth Shostak, and 9 minutes long. I talk about ems, naturally. The other three guests are Luke Muehlhauser, Stuart Newman, & Annalee Newitz.

It is a pretty good show; I think I was told it has 150,000 listeners. They say they are “supported in part by Sami David, Rena Shulsky David, and the NASA Astrobiology Institute.” My thanks to them as well.

GD Star Rating
loading...
Tagged as: ,

Imagine Farmer Rights

Yesterday I criticized proposals by George Dvorsky and Anders Sandberg to give rights to ems by saying that random rights are bad. That is, rights limit options, which is usually bad, so those who argue for specific rights should offer specific reasons why the rights they propose are exceptional cases where limiting options helps strategically. I illustrated this principle with the example of a diner’s bill of rights.

One possible counter argument is that these proposed em rights are not random; they tend to ensure ems can keep having stuff most of us now have and like. I agree that their proposals do fit this pattern. But the issue is whether rights are random with respect to the set of cases where strategic gains come by limiting options. Do we have reasons to think that strategic benefits tend to come from giving ems the right to preserve industry era lifestyle features?

To help us think about this, I suggest we consider whether we industry era folks would benefit had farmer era folks imposed farmer rights, i.e., rights to ensure that industry era folks could keep things most farmers had and liked. For example, imagine we today had “farmer rights” to:

  1. Work in the open with fresh air and sun.
  2. See how all  food is grown and prepared.
  3. Nights outside are usually quiet and dark.
  4. Quickly get to a mile-long all-nature walk.
  5. All one meets are folks one knows, or known by them.
  6. Easily take apart devices, to see materials, mechanisms.
  7. Authorities with clear answers on cosmology, morality.
  8. Severe punishment of heretics who contradict authorities.
  9. Prior generations quickly make room for new generations.
  10. Rule by a king of our ethnicity, with clear inheritance.
  11. Visible deference from nearby authority-declared inferiors.
  12. More?

Would our lives today be better or worse because of such rights?

Added: I expect to hear this response:

Farmer era folks were wrong about what lifestyles help humans flourish, while we industry era folks are right. This is why their rights would have been bad for us, but our rights would be good for ems.

GD Star Rating
loading...
Tagged as: , , , ,

Random Rights Are Bad

Food truck and fast food meals can be pretty skimpy. So wouldn’t it be great if we passed a diner’s bill of rights law, requiring all prepared food to come with free unlimited drinks, a fast human waiter, cloth napkins and tablecloths, and a seat by a window? Well no, that wouldn’t be great. All those food trucks and fast food places would go out of business, leaving diners only with the option to eat at expensive fancy restaurants.

It might feel good to play Santa for free, handing out stuff that costs you little yet appears to benefit others lots. But something-for-nothings are usually illusions. Rights limit options, and that is generally bad.

Yes, sometimes we can benefit strategically from having our options limited, but such situations are rare. Random limits on options are usually bad. So if you propose limiting options, you should be prepared to offer particular arguments for why your particular cases are in fact strategic exceptions.

George Dvorsky says we should give lots of rights to ems:

If we’re going to be making minds, we sure as hell need to do it responsibly. … This was the topic of Anders Sandberg’s talk … about the harm that could be inflicted on software capable of experiencing thoughts, emotions, and sensations. … Sandberg proposed that virtual [lab] mice be given virtual painkillers. Another issue is time-rate rights. Does a human emulation have the right to live in real-time, so that it can interact properly with non-digital society? …

Back in 2010, … I proposed that the following rights be afforded to fully conscious human and human-like emulations:

  • The right to not be shut down against one’s will
  • The right to not be experimented upon
  • The right to have full and unhindered access to one’s own source code
  • The right to not have one’s source code manipulated against their will
  • The right to copy (or not copy) oneself
  • The right to privacy (namely the right to conceal one’s own internal mental states)
  • The right of self-determination

… I’d like to include Sandberg’s idea of time-rate rights.

In the comments Dvorsky also likes a “right to have a body and senses.”

But just as with a diner’s bill of rights, limiting options is in general bad. Ems would usually choose each new life, by negotiating with employers, landlords, etc. for a job, place, etc. for a new copy to live. So just as requiring free drinks with a meal can take away that meal as an option, requiring an em life to come with “a right to live in real time” may take away that life as an option. Since the cost to run an em is roughly linear in speed, prohibiting ems that can’t run faster than a thousand times slower than human speeds can in effect raise the cost of such ems by a factor of a thousand. That might greatly reduce the demand for such ems, and hence their number.

Yes, there may be particular situations where limiting options helps ems, but we should expect to hear arguments for why particular cases are exceptions to the usual rule. Dvorsky offers no such arguments, and given how little we know about the em world it is hard to believe he’s worked out detailed arguments that he forgot to mention. Maybe Dvorsky just likes to play Santa for free?

Btw, here’s a post where I criticize a similar effort by Greg Egan to give ems rights. See also Alex on harmful rights.

Added: Just as we have good reasons to stop people from being forced to eat meals that they didn’t choose, we also have good reasons to stop ems from being forced to live lives they didn’t choose. We know lots about why property rights are often useful. More examples of bad random rights:

  • The right to a bookstore that has certain random books.
  • The right to a kitchen holding certain random spices and utensils.
  • The right to a home with certain random furniture items.
  • The right to a movie with certain random plot elements.
  • The right to a laptop with certain random features and accessories.

Added 10a: Anders’ talk is based on this paper; key quote: Continue reading "Random Rights Are Bad" »

GD Star Rating
loading...
Tagged as: ,

Bits Of Secrets

“It’s classified. I could tell you, but then I’d have to kill you.” Top Gun, 1986

Today, secrets are lumpy. You might know some info that would help you persuade someone of something, but reasonably fear that if you told them, they’d tell others, change their opinion on something else, or perhaps just get discouraged. Today, you can’t just tell them one implication of your secret. In the future, however, the ability to copy and erase minds (as in am em scenario) might make secrets much less lumpy – you could tell someone just one implication of a secret.

For example, what if you wanted to convince an associate that they should not go to a certain party. Your reason is that one of their exes will attend the party. But if you told them that directly, they would then know that this ex is in town, is friendly with the party host, etc. You might just tell them to trust you, but what if they don’t?

Imagine you could just say to your associate “I could tell you why you shouldn’t go to the party, but then I’d have to kill you,” and they could reply “Prove it.” Both of your minds would then be copied and placed together into an isolated “box,” perhaps with access to some public or other info sources. Inside the box the copy of you would explain your reasons to the copy of them. When the conversation was done, the entire box would be erased, and the original two of you would just hear a single bit answer, “yes” or “no,” chosen by the copy of your associate.

Now, as usual, there are some complications. For example, the fact that you suggested using the box, as opposed to just revealing your secrets, could be a useful clue to them, as could the fact that you were willing to spend resources to use the box. If you requested access to unusual sources while in the box, that might give further clues.

If you let the box return more detail about their degree of confidence in their conclusion, or about how long the conversation took, your associate might use some of those extra bits to encode more of your secrets. And if the info sources accessed by those in the box used simple cacheing, outsiders might see which sources were easier to access afterward, and use that to infer which sources had been accessed from in the box, which might encode more relevant info. So you’d probably want to be careful to run the box for a standard time period, with unobservable access to standard wide sources, and to return only a one bit conclusion.

Inside the box, you might just reveal that you had committed in some way to hurt your associate if they didn’t return the answer you wanted. To avoid this problem, it might be usual practice to have an independent (and hard to hurt) judge also join you in the box, with the power to make the box return “void” if they suspected such threats were being made. To reduce the cost of using these boxes, you might have prediction markets on what such boxes would return if made, but only actually make them a small percentage of the time.

There may be further complications I haven’t thought of, but at the moment I’m more interested in how this ability might be used. In the world around you, who would be tempted to prove what this way?

For example, would you prove to work associates that your proposed compromise is politically sound, without revealing your private political info about who would support or oppose it? Prove to investigators that you do not hold stolen items by letting them look through your private stores? Prove to a date you’ve been thinking lots about them, by letting them watch a video of your recent activities? Prove to a jury of voters that you really just want to help them, by letting them watch all of your activities for the last few months? What else?

In general, this would seem to better enable self-deception. You could actually not know things anywhere in your head, but still act on them when they mattered a lot.

GD Star Rating
loading...
Tagged as: , , , ,

Robot Econ Primer

A recent burst of econo-blog posts on the subject of a future robot based economy mostly seem to treat the subject as if those few bloggers were the only people ever to consider the subject. But in fact, people have been considering the subject for centuries. I myself have written dozens of posts just here on this blog.

So let me offer a quick robot econ primer, i.e. important points widely known among folks who have long discussed the subject, but often not quickly rediscovered by dilettantes new to the subject:

  • AI takes software, not just hardware. It is tempting to project when artificial intelligence (AI) will arrive by projecting when a million dollars of computer hardware will have a computing power comparable to a human brain. But AI needs both hardware and software. It might be that when the software is available, AI will be possible with today’s computer hardware.
  • AI software progress has been slow. My small informal survey of AI experts finds that they typically estimate that in the last 20 years their specific subfield of AI has gone ~5-10% of the way toward human level abilities, with no noticeable acceleration. At that rate it will take centuries to get human level AI.
  • Emulations might bring AI software sooner. Human brains already have human level software. It should be possible to copy that software into computer hardware, and it seems likely that this will be possible within a century.
  • Emulations would be sudden and human-like. Since having an almost emulation probably isn’t of much use, emulations can make for a sudden transition to a robot economy. Being copies of humans, early emulations are more understandable and predictable than robots more generically, and many humans would empathize deeply with them.
  • Growth rates would be much faster. Our economic growth rates are limited by the rate at which we can grow labor. Whether based on emulations or other AI, a robot economy could grow its substitute for labor much faster, allowing it to grow much faster (as in an AK growth model). A robot economy isn’t just like our economy, but with robots substituted for humans. Things would soon change very fast.
  • There probably won’t be a grand war, or grand deal. The past transitions from foraging to farming and farming to industry were similarly unprecedented, sudden, and disruptive. But there wasn’t a grand war between foragers and farmers, or between farmers and industry, though in particular wars the sides were somewhat correlated. There also wasn’t anything like a grand deal to allow farming or industry by paying off folks doing things the old ways. The change to a robot economy seems too big, broad, and fast to make grand overall wars or deals likely, though there may be local wars or deals.

There’s lots more I could add, but this should be enough for now.

GD Star Rating
loading...
Tagged as: , ,

Cooperate Or Specialize?

Futurists sometimes get excited about new ways to encourage cooperation in Prisoner’s Dilmena like games. For example, future folks might interact via quantum games, future AIs might show each other their source code, or future clans of em copies might super-cooperate with one another. Folks who know just enough economics to be dangerous sometimes say that this “changes everything”, i.e., that future economies will be completely different as a result. In fact, however, not only do we already have lots decent ways to encourage cooperation, such as talking and reputation, we also consistently forgo such ways to better encourage flexibility and specialization.

As I reviewed in my last post, we have strong reasons and abilities to cooperate within family clans, especially when such clans heavily intermarry and live and work closely together over many generations. And our farming era ancestors took big advantage of this. To function and thrive, however, our industry era economy had to suppress such clans, to allow more flexibility and specialization. Industry needs people to frequently change where they live, what kinds of jobs they do, and who they work with, and to play fair within industry-era reimagined firms, cities, and nations. Strong family clans instead encouraged stability and nepotism, and discouraged people from moving to cities and new jobs, and from cooperating fairly with and showing sufficient loyalty to other families within shared firms, cities, and nations.

Our industry era institutions consistently forgo the extra cooperation advantages of strong family clans, to gain more flexibility and specialization. This is now a huge net win. Our descendants are likely to similarly forgo advantages from new ways to cooperate, if those similarly reduce future flexibility and specialization. For example, future societies of brain emulations are likely to be wary of strongly self-cooperating clans of copies of the same original human. While such copy clans have even stronger reasons to cooperate with each other than family clans, copy clans might cause future organizations to suffer even more than do family-based firms, cities, and nations today from clan-based nepotism, and from low quality and inflexible matches of skills to jobs. Ems firms and cities are thus likely to be especially watchful for clan nepotism, and to avoid relying too heavily on any one clan.

Yes game theory captures important truths about human behavior, including about costs we pay from failing to fully cooperate. But prisoner’s dilemma style failures to cooperate in simple games comprises only a tiny fraction of all the important things that can and do go wrong in a modern economy. And we already have many decent ways to encourage cooperation. I thus conclude that future economies are unlikely to be heavily redesigned to take advantage of new possible ways to encourage prisoner’s dilemma style cooperation.

GD Star Rating
loading...
Tagged as: , , , ,

Estonia Talk Friday

This Friday I’ll speak on Em Econ at TEDx Tallinn 2013, in Estonia. You can apparently see it online if you organize and register a “live simulcast event.” A few talks from prior years are online to watch, but I have no idea if my talk will become one of those.

Added: Slides, Video.

GD Star Rating
loading...
Tagged as: ,

Em Econ At Harvard

I’ll talk on Em Econ twice this week at Harvard:

Added: The Boston lockdown Friday moved the discussion to Saturday. I’ve added links to audio and slides.

GD Star Rating
loading...
Tagged as: , ,

A City Is A Village Of Villages

There have been three major eras of human history: foraging, farming, and industry. During each era our economy has grown at a roughly steady exponential rate, and I’ve written before about some intriguing patterns in these growth eras: eras encompassed a similar number of doublings (~7-10), transitions between eras were much shorter than prior doubling times, and such transitions encompassed a similar number of growth rate doublings (~6-8). I’ve also noted that transition-induced inequality seems to have fallen over time.

I just noticed another intriguing pattern, this time in community sizes. Today in industrial societies roughly half of the population lives in metropolitan areas with between one hundred thousand and ten million people, with a mid size of about a million. While good data seems hard to find, during the farming era most people seem to have lived in communities (usually centered around a village) of between roughly three hundred and three thousand people, with a mid size of about a thousand. Foragers typically lived in mobile bands of size roughly twenty to fifty, with a best size of about thirty.

So community sizes went roughly from thirty to a thousand to a million. The pattern here is that each new era had a typical community size that was roughly the square of the size during the previous era. That is, a city is roughly a village of villages, and a village is roughly a band of bands. We could extend this patter further if we liked, saying that an extended family group has about four to eight members, with a mid size of six, so that a band is a family of families. (We might even go further and say that a family is a couple of couples, where a couple has two or so members.)

If previous growth patterns were to continue, I’ve written before that a new growth era might appear sometime in roughly the next century, and over a few years the economy would transition to a new growth rate of doubling every week to month. If this newly-noticed community size pattern were to continue, the new era would have communities of size roughly a trillion, perhaps ranging from ten billion to a hundred trillion.

Admittedly, after a year or two of this new era, things might change again, to yet another era. And the growth and community size trends couldn’t both continue to that next era, since a community size of a trillion trillion would require much more than twenty doublings of growth. So these trends clearly have to break down at some point.

I’ve been exploring a particular scenario for this new era: it might be enabled and dominated by brain emulations, or “ems.” Interestingly, I had already estimated an em community size of roughly a trillion based on other considerations. Ems could take up much less physical space than do humans, and since ems could visit each other in virtual reality without moving physically, em community sizes would be less limited by travel congestion costs.

So what should one call a city of cities of a trillion souls? A “world”?

GD Star Rating
loading...
Tagged as: , , ,