Discussion about this post

User's avatar
Peter Gerdes's avatar

This is one of the reasons that a belief in FOOM (fast AI takeoff) is incompatible with the standard arguments the AI alignment problem is hard or even impossible. Digital minds need to worry about drift as well.

If the problem is hard then we should expect it will be hard for the first super human AI we create. The arguments that we face an alignment problem depend on establishing that this AI will have goals it wants to achieve (the concern is they differ from ours) but that AI will then refrain from creating improved versions of itself because it won't be able to ensure they are aligned with it.

Expand full comment
Michael J. McGuffin's avatar

"digital minds [...] could succumb to poverty, disease, and war" Aren't these easily prevented in a virtual world, especially disease?

Expand full comment
21 more comments...

No posts