The framework of ‘niches’ encompasses variation in idiosyncratic specialization, and pure benefits of cognitive diversity. The benefits of specialization and diversity mean that there will be many niches, but will not protect against turnover in those niches.

jb,

Yes, em populations might be located on physically isolated hardware as a means of social control, although boxing without monitoring might lead to regrets.

And then I would unleash a zombie virus upon them and cackle maniacally from my dark throne!

This is certainly arguable.

Experts who make predictions, for example – two copies of the best weather predicter in the world will make the same predictions, and be no better than one. A second expert who doesn’t predict correctly as often as the best predicter, but who makes INDEPENDENT errors, is the one to add to your team.

The majority of the German population, critically including the German military and SS, did not expect to be targeted in the Holocaust. With their acceptance of Nazi authority, Jewish/Roma/gay insurrection was visibly very unlikely to succeed.

“Apologies for misinterpreting this. I would assume that any EM smart enough to recognise this problem will not seek to rent its hardware or energy supply, but rather, to buy it.”

If this is done at the individual level, then its labor will be much more expensive than cheap renter ems, which will outcompete it. As Robin notes in his uploads paper, there will be evolutionary pressures for willingness to create copies who will face poor and risky prospects as renters. However, there will be much weaker pressures to make those ems willing to submit to mass eviction.

“What can’t be allowed won’t be allowed. If EMs are as smart as the people on this thread then presumably they won’t allow Malthusian growth.”
‘Draconian measures,’ i.e. an em singleton.

Eriol,

Merging ems would require a deep understanding of the brain, whereas the emulation scenario assumes the lack of such. If you have that level of understanding you also get customized AIs.

>It’s not my idea: Robin explicitly says that in his preferred scenario cheap ems will rent their hardware and be subject to lethal
>eviction. I am critiquing his scenario, arguing that it would be more unstable than Robin seems to believe.

Apologies for misinterpreting this. I would assume that any EM smart enough to recognise this problem will not seek to rent its hardware or energy supply, but rather, to buy it. It will also co-operate with other EMs to pay a large number of Lobbyist programs to lobby government against allowing unrestricted proliferation of newer better AIs.

In other words I’m saying EMs will become GMs.

>”Besides, with an army of AIs, we can expect conquering the universe, almost free energy etc. to come along too.”
>Malthusian growth with replication times measured in hours or days will exhaust available resources very quickly, and lightspeed >limits restrict acquisition of resources via space colonization to a geometric (cubic) pattern.

What can’t be allowed won’t be allowed. If EMs are as smart as the people on this thread then presumably they won’t allow Malthusian growth.

Lightware:> Wouldn’t people start feeling useless and worthless if ems are better at everything?

As an above-average AI researcher who reads Eliezer’s posts, I can say with some certainty, yes, they probably will

Anonymous.

There was even an estimate somewhere on Wikipedia that the whole Holocaust cost Nazis a few hundred of their own dead – millions just complied with being genocided. In less extreme situations there would be even less resistance.

We see less of such abuse right now, but mostly because there’s such an abundance of almost everything, not because human nature somehow changed. If one day scarcity returns people won’t even remember all the equality stuff.

Don: I’ve been reading O.B. for most of its existence, and have read everything from the very start (and I might as well say here, it’s the most intellectually satisfying blog I have ever read: even the comments are mostly worth reading). Yes, I remember Eliezer’s digression into QM, but he did eventually tie it back to o.b.