Discussion about this post

User's avatar
Overcoming Bias Commenter's avatar

I think in the discussion above, there is a lot of conflation between causes for what is termed "total tech war." You can find yourself in a total tech war merely by believing that the other agents see it as such. Or you can independently analyze the situation and determine that the best way to maximize your own payoff is to treat it as a total tech war regardless of what the other agents will think about it. If the space of advantages as upward cliffs as Eliezer suggests, then it is not unreasonable to believe an agent with a time sensitive, but utterly dominating, advantage will rationally decide the most payoff happens by acting in accordance with a total tech war plan of action. This is especially true if part of the cliff-advantage is the ability to analyze a situation more deeply and rapidly than competitors. I don't see any reason why extra, special arguments are needed to justify this as a realistic scenario within AI FOOM.

Expand full comment
Tim Tyler's avatar

if you want to say that a particular tech is more winner take all than usual, you need an argument based on more than just this effect. And if you want to argument it is far more so than any other tech humans have ever seen, you need a damn good additional argument.

IT is the bleeding edge of technology - and is more effective than most tech at creating inequalities - e.g. look at the list of top billionaires.

Machine intelligence is at the bleeding edge of IT. It is IT's "killer application". Whether its inventors will exploit its potential to provide wealth will be a matter of historical contingency - but the potential certainly looks as though it will be there. In particular, it looks as though it is likely to be mostly a server-side technology - and those are the easiest for the owners to hang on to - by preventing others from reverse-engineering the technology.

Expand full comment
21 more comments...

No posts