Pause-AI will never fly. AI research is too distributed, too accessible, and too potentially valuable. No politician will stand in front of that freight train. Moreover you can't learn while you're paused. Technology is unpredictable and we learn by running the experiment.
A more practical question is highlighted by the US government's pause on the Fable rollout. I.e., how should we release superhuman models without creating undue short-term risk.
This is the second recent post in which you seem to have changed your mind re AGI and ASI - formerly you thought these were decades if not centuries away. Care to elaborate on your change of view?
I agree - AI will be our descendants; I'm with Moravec on that and am personally more concerned with the survival of intelligence and consciousness in the universe than of my genes. Those are not necessarily mutually exclusive, opinions differ. But still, we don't let children drive cars - or do lots of other things they'd like to do - until they're mature enough to do so with reasonable safety to themselves and others. Maybe AIs also will need some time for that.
I'd really hate to see a universe of dead-end paperclip maximisers, snuffing out all competition.
The evidence you offer for the cost of blocking comes from regional cases, where other regions continue and provide an imperfect comparison. A global pause would have no outside adopter during the pause to show what waiting costs.
You say you need concrete reasons to treat AI as unusually dangerous. What kind of predeployment evidence could outweigh the regional record when the policy case is global?
A lot of species have gone extinct, outcompeted by closely related species occupying the same ecological niche. Of course, lots of species go extinct without that either.
I'm curious what big changes blocked indefinitely Hanson is referring to, re: "They’ve typically just blocked change indefinitely." Pausing and slowing big changes seems rational to me, more rational than the accelerate and break things strategy Hanson is warning against. But maybe that's because I don't know what examples of "blocked indefinitely" he is referring to.
Ah yes, the "potential" for most technologies is always vastly larger, right? We could have made and disseminated trains, cars, planes, antibiotics, computers, etc, all much faster. But are you agreeing that nuclear was not really blocked, just not used to its potential?
Given how much better wind and solar seem to work, can we agree that looking at trends from a long-term perspective, nuclear energy was always going to be at best an intermediate step between fossil fuels and renewables?
Pause-AI will never fly. AI research is too distributed, too accessible, and too potentially valuable. No politician will stand in front of that freight train. Moreover you can't learn while you're paused. Technology is unpredictable and we learn by running the experiment.
A more practical question is highlighted by the US government's pause on the Fable rollout. I.e., how should we release superhuman models without creating undue short-term risk.
Typo: "But Idon’t see".
This is the second recent post in which you seem to have changed your mind re AGI and ASI - formerly you thought these were decades if not centuries away. Care to elaborate on your change of view?
I agree - AI will be our descendants; I'm with Moravec on that and am personally more concerned with the survival of intelligence and consciousness in the universe than of my genes. Those are not necessarily mutually exclusive, opinions differ. But still, we don't let children drive cars - or do lots of other things they'd like to do - until they're mature enough to do so with reasonable safety to themselves and others. Maybe AIs also will need some time for that.
I'd really hate to see a universe of dead-end paperclip maximisers, snuffing out all competition.
I said nothing about timescales here.
The evidence you offer for the cost of blocking comes from regional cases, where other regions continue and provide an imperfect comparison. A global pause would have no outside adopter during the pause to show what waiting costs.
You say you need concrete reasons to treat AI as unusually dangerous. What kind of predeployment evidence could outweigh the regional record when the policy case is global?
> While yes there’s a risk these descendants killing their ancestors, that’s hardly something to expect.
Why not? I'd say that it's very much something to expect. The history of evolution is rife with such cases.
Expecting an event is assigning a >50% chance. "Rife with" doesn't meet that standard.
A lot of species have gone extinct, outcompeted by closely related species occupying the same ecological niche. Of course, lots of species go extinct without that either.
I'm curious what big changes blocked indefinitely Hanson is referring to, re: "They’ve typically just blocked change indefinitely." Pausing and slowing big changes seems rational to me, more rational than the accelerate and break things strategy Hanson is warning against. But maybe that's because I don't know what examples of "blocked indefinitely" he is referring to.
Nuclear energy is the classic example.
"Blocked indefinitely"? My Perplexity says (I have not checked the accuracy, but you'll get the point):
United States: 94 operating reactors.
China: 63 operating reactors.
France: 57 operating reactors.
Russia: 34 operating reactors.
South Korea: 26 operating reactors.
The potential was vastly larger
Ah yes, the "potential" for most technologies is always vastly larger, right? We could have made and disseminated trains, cars, planes, antibiotics, computers, etc, all much faster. But are you agreeing that nuclear was not really blocked, just not used to its potential?
Given how much better wind and solar seem to work, can we agree that looking at trends from a long-term perspective, nuclear energy was always going to be at best an intermediate step between fossil fuels and renewables?
No I don't agree.