90 Comments

I just added to this post.

Expand full comment

Re: I rather doubt that ecosystems of diverse artifical organisms can be created by man to replace lost ones (and I’m biologically well educated). If they can be, good – but how do we know?

Look at the universe of software - where there are many different types of program, each with many copies. That is roughly the level of diversity to be expected in ecosystems created by man - and we can expect something very similar once we have some more decent robots.

Expand full comment

The evolutionary process works to minimise conflict - because it is a wasteful and inefficient use of resources. Thus we see large numbers of mock battles, lots of signalling intended to avoid direct conflict, and the virtualisation of warfare - as seen in the arenas of business and sport.

Real conflicts still happen - but the evolutionary process is not done yet. Once the software/hardware divide permeates nature, things will improve further.

Expand full comment

There has been quite a bit of competition over the past 3.7 billion years - it has certainly been a stable state so far. Any agents with big power advantages in the past have tended to reproduce - and then their descendants diverged and competed among each other.

Expand full comment

The "dumb nanobot" scenario is incredibly unlikely - and the "dumb nanobot that can't evolve" scenario is even more ridiculous. Do we really need to spend time considering such thought experiments?

Expand full comment

I think that there is an implicit agreement that when Robin is presented with a sufficiently-detailed-to-be-understandable description of a particular upload life that I deem as "inhumane", he will choose. I may do an OB post to describe such a scenario.

Expand full comment

I refused to make a choice I when I didn't understand the options.

Expand full comment

@Wei: Well, if you do that, I'm probably going to respond by pointing out that Robin refused to choose at the point where Roko pressed him to specifically endorse a particular inhumane outcome or else deny that it would be an acceptable outcome of competition. Which suggests that Robin has some measure of allegiance to competition, but something like ordinary human morality alongside it, and so would lose either way if forced to choose even hypothetically.

Expand full comment

"preference for weaker species, which creates a weaker ecology."

How is human interest related to the survival strength of a species? It is more related to novel mutations (tangible utility), cuteness (less-tangible utility), and possibly sympathetic preference for life over death. Taking action to preserve these things is in human interest. Nature-lover ideology often refers to the cuteness factor, but it also references human utility (i.e. from whatever rare species of undiscovered amazonian frog). Humans naturally interact with these species (i.e. by chopping down forests and building roads) which can take away from human utility (by killing off undiscovered cancer-curing frogs and replacing them with crows, which we already have a bunch of).

Saying that "building roads is bad" is different from saying "previously building roads gave us high utility, but now the utility of not-building is potentially higher because cancer-curing frogs might live in those roads."

Human behavior changes because conditions change. This is (I hope?) the contradiction jonathan referred to.

i.e. in the past, there were no limits on businesses and lots of competition. Now, there is less competition because big businesses can squash it. Since conditions have changed (the size of businesses and their power) maybe a different response (regulation of business power) will create more utility (competition among coffee shops rather than 4 Starbucks on one block that are all closed at 9 pm on Saturday).

Expand full comment

@Robin:

Sure, perhaps a comment thread is not the best place to thrash out the true nature of our difference. I'll see you at the Summit!

Expand full comment

@RichardHollerith:

development that if successful will completely transform the world and probably make choices that no one in the lightcone will ever be able to reverse.

Choosing to not act is itself a choice, and supporting competition means supporting certain political stances, including implicitly advocating a future choice to unleash the uploads, which would probably be hard or impossible to reverse. Remember, the time between uploads and the next phase shift to a faster doubling regime is hypothesized - by Robin - to be less than a year (correct me if I am wrong here).

Expand full comment

@Eliezer: Well, I guess I have demonstrated the ability to make a mistake once rather than twice ;-0

Expand full comment

Weak or ambiguous property rights could be preventing the development of regions, which if developed would drive species extinct.

Expand full comment

Robin is neither enthusiastic promoter of or wannabe captain in a "Napoleon's army" type of development that if successful will completely transform the world and probably make choices that no one in the lightcone will ever be able to reverse. Consequently, there is no need for Robin to publish a definition of his morality detailed enough to cover a world completely transformed.

Napoleon's army was the target of idealistic hopes and the best apparent chance for a large fraction of the bright ambitious young Frenchmen to gain social status in French society.

Expand full comment

What kind of market failure could possibly save a species that should have been lost? The only kind of failure of group rationality that I can imagine might cause that is a government failure. (The other direction, that markets will fail to preserve some species that should be preserved, seems obvious, since species (or information embodied by species) are not owned, so it's a typical tragedy-of-the-common scenario.)

If you agree that markets can only fail in one direction on this issue, then the optimal amount of government intervention probably isn't zero (even taking into account the possibility of government failure).

Expand full comment

Roko, it would take us a lot of effort to figure out what exactly you mean by your no friends, no morals, etc. scenario, before I could place probabilities and relative values on it. This comment thread probably isn't the best place to work that out.

Expand full comment