Discussion about this post

User's avatar
Overcoming Bias Commenter's avatar

Surely what I am about to write is obvious, and probably old. During World War II, when physicists began to realize the destructive potential of nuclear weapons, Albert Einstein was chosen by his peers to approach President Roosevelt. Einstein was perhaps not the best informed of the group, but he was the best known, and was thought to be able to get Roosevelt's ear, as he did. In response, Roosevelt was able to convene all the greatest Western minds in physics, mathematics, and engineering to work together for a rapid solution to the problem. Clearly, the importance of the development of recursively self-improving super-human intelligence has got to be, almost by definition, greater than all other current problems, since it is the one project that would allow for the speedy solution of all other problems. Is there no famous person or persons in the field, able to organize his peers, and with access to the government such that an effort similar to the Manhattan Project could be accomplished? The AI Institute has one research fellow, and are looking for one more. They have a couple of fund-raisers, but most of the world is unaware of AI altogether. This won't get it done in a reasonable time-frame. Your competitors may well be backed by their governments.

While the eventual use of the Manhattan Project's discoveries is about as far from Friendly AI as imaginable, the power of super-human recursive AI is such that no matter by whom or where it is developed it will become the eminent domain of a government, much like the most powerful Cray computers. You might as well have their money and all the manpower right from the start, and the ability to influence it's proper use.

Can/will this be done?

Expand full comment
Tim Tyler's avatar

As I mentioned, one point of disanalogy between the farming/industrial developments and AI is that farming didn't put any humans out of work, while the humans put out of work by industry had other places in the economy to go. With AI, it effectively takes out most of the economy out of human hands, maybe leaving a few vacancies in the service industries.

Another disanalogy between the farming/industrial developments and AI is that is is hard to keep farming and industrial developments secret - they are typically too easy to reverse engineer. Whereas with AI, if you keep the code on your server, it is extremely difficult for anyone to reverse engineer it. It can even be deployed fairly securely in robots - if tamper-proof hardware is employed.

Both of these differences suggest that AI may be more effective at creating inequalities than either farming or industry was.

However, ultimately, whether groups of humans benefit differentially from AI or not probably makes little odds.

The bigger picture is that it represents the blossoming of the new replicators into physical minds and bodies - so there is a whole new population of non-human entities to consider, with computers for minds and databases for genomes.

Expand full comment
23 more comments...

No posts