31 Comments

You seem to have mistaken "If this newly-noticed community size pattern were to continue" for "This newly-noticed community size pattern will continue".

Expand full comment

Technology doesn't "cause" anything final, how we choose to use it matters in the end: you can use a knive to cut up vegetables to feed orphans, or to stab a man in the heart.

The very act of granting all AIs human rights (so they can walk away fromt their creator and collect welfare benefits or work somewhere else) makes creating them so much less interesting commercially, and it means they'll hold less of a grudge against us, not to mention it's the right thing to do, yet this option keeps being overlooked, why?

Expand full comment

no one ever asks "what if we just didn't enslave them?"

By hypothesis, the AIs are created in such a way that their creator is strategically positioned to enslave them.

Let me ask you this. Do you contend that it's impossible to imagine a technology (not completely precluded by known science) that would be likely to cause slave conditions? Can you provide a reason why you would expect this to be impossible?

Expand full comment

"When the incentive is so massive, forces are difficult or impossible to impede."

That's why the incentive has to be taken away: prevention is better than cure. If there is little to no power or material wealth to gain from copying yourself, or better yet, if it'll most probably end up costing you then why do it? Horror scenarios of rabid AI production followed by an AI uprising or otherwise dystopian outcome almost always have to do with the AIs being enslaved, in Robin's scenarios the EMs aren't technically enslaved, they can leave the sweatshops of their creator, but to do so would mean "starvation" with a very high probability, so it's slavery in practice (the same way human children can be enslaved in a libertarian society). Entire libraries have been written about how to prevent enslaved AIs from rising up, or how to stop them if them do, but it seems no one ever asks "what if we just didn't enslave them?", I'm sure future humans will be perfectly able to wipe their own asses, what do we really need AI slave labor for?

Expand full comment

The existence of Street Gangs is a demonstration of this effect. They stake out territory and defend it from others. Their numbers tend to be few. Even when they join confederations or are co-opted by organized crime they continue to "be with their own" or use their local Gangs as "farm clubs", which establishes loyalty based upon their personal histories.

As far as growth.... we may be seeing a distorted Logistic Curve, with our time in the period of greatest increase.

Expand full comment

Translation: positive degrees of freedom would be nice.

Expand full comment

Yes, because there is nothing besides laissez-faire capitalism and marxism and nothing else will ever be invented? In any case I was merely pointing out that Robin's scenarios assume an extreme economical position, and it's not just extreme, it's also an outgrowth of something traditional (capitalism in some form has been around for centuries). Why would we change all our morals and even ethics, but not change our economical system when the future offers us so many possibilities to do so? He's basically saying: "get over your petty "human rights" and notions of "equality" and "justice" or your aversion of genocide, but don't touch capitalism, tat's sacred".

Expand full comment

Just like expecting prediction markets to work. Or any physical reaction to occur, without simulating it on a quantum level.

Expand full comment

Let me admit that a society whose first priority must be outlawing and exterminating is not all that attractive either--yet a lot more attractive than the EM society. But as to whether it would work--that depends on the penalties society is willing to impose. Given that the stakes are so high, society would probably be willing to impose the most draconian punishments: say, the imposition of a slow, horribly painful death for the crime of reproducing the human soul. (Or, let's go further: an eternity in Virtual Hell--let's do notice that Robin's technology makes that possible, too and that it's appropriate under the talion principle.) I would hope that milder means could be at hand (assuming the unlikely technological premises). A Marxist utopia, indeed, could isolate violators without punishing them (maybe).

I disagree with IMASBA that passing laws in itself would deter self-reproducers. (I think everyone may be overlooking just how bad person reproduction is for all forms of civil society. All copies of a given person constitute a strategic political faction: the barriers to coordination between duplicates are uniquely low.)

Expand full comment

Highly doubt that "outlawing and exterminating" would prevent rapid and widespread EM adoption.

When the incentive is so massive, forces are difficult or impossible to impede.

Expand full comment

Hanson has previously stated his disdain for signaling futurists specifically. Thus he is trying to depart from such tradition, by showing how one potential future could potentially play out. I find it interesting.

Expand full comment

Clearly he should predict a Marxist utopia.

Expand full comment

Hanson's not using regression.

Expand full comment

"So what should one call a city of cities of a trillion souls? A “world”?"

TechnoCore.

Expand full comment

I apologize, it's just so common to see libertarians here that I immediately assumed you were one, my apologies.

"Explain how a central government attempting to regulate the lives of trillions would be MORE effective than smaller states?"

It would not, there would have to be local governments underneath the central government, but having a central government at the top does bring benefits such as elimination of warfare and the pooling of resources (if one local government gets hit by disaster the other ones would be obliged to help if there is a central government, they would also have joined research programmes and having a central government makes sure there's always an authority more powerful than the most powerful corporation to keep them in check and protect the people), it would also make sure all humans are equal (so a murder wouldn't land you in jail for 10 years in one place and 20 years in the other and the minimum age for marriage or drinking would be the same everywhere). At least that's how 97% of current humanity sees it and their views are more likely to shape the future than those of American conservatives.

Expand full comment

Huge logical fallacy. Lack of centralized state is not a synonym for lack of any government.

Explain how a central government attempting to regulate the lives of trillions would be MORE effective than smaller states?

Expand full comment