AI Pause Regs Look Risky

People dislike AI for many reasons, but one extreme fear is that AIs will escape control from those who make, buy, or rent them, and then hurt others. We usually deal with such problems via legal liability, when law is available, or via threats of retaliation, when law is not available. But while those work among communities of comparably powered agents, some fear that out-of-control AIs might eventually get powerful enough and coordinate well enough to basically take over the world, and then enslave or end the rest of us.

To prevent this scenario, many want us to now impose strong global regulations on powerful AIs, limiting their powers, who can make or use them, testing to see which ones can or try to escape, and what ends escaped AIs pursue. Some call this regulation a “pause”, as if it were likely that we would soon find reliable solutions, after which regulations could greatly ease. That seems misleading, as we have little reason to think we are close to dramatic gains in our AI control abilities. So if we regulate for as long as regulators continue to express concern, regulation could last quite a long time.

Most advocates of such regulation now see only a small number of identifiable and concentrated “frontier AI firms” as making AIs powerful enough to raise these concerns. So they propose to only regulate those for now. For example, some say only Chinese and US firms are big enough, so a treaty between them would suffice.

But AI costs to answer a set of fixed but hard questions seems to have fallen roughly ~x10 (in x3-x30) per year over the last four years. And so the number of nations, firms, and orgs who can afford to make, buy, or rent powerful AI is rising fast. This implies that in order to hold constant levels of regulation regarding AIs with particular levels of power, a regulatory regime would have to expand in scope quite fast, and without obvious limit. While at first only specialized AI chips and data-centers might need to be monitored, soon other kinds could also be used to make powerful AIs.

Furthermore, there isn’t a natural clear dividing line between AI and other kinds of software. Sure the most advanced AIs that we have now tend to have particular structures, and so regulations could target only software with such structures. But a vast space of possible structures remains possible and even promising. And the core concept here of systems with powerful ways to help humans do stuff could plausibly cover most software.

We have long seen many regulatory agencies slowly expand their scope, even in contexts of constant public opinion and relevant technology. So there is a real risk here that an AI regulatory agency would expand its scope to cover most computer software and hardware.

Some have proposed AI regs that would focus only on the most powerful AI systems at any one time, and as a result relax regulation on given-power-level AIs over time. This might make sense if the risk is only that the most powerful AIs might escape control; less powerful ones might be restrained by the more powerful. But even here the scope of regulation would rapidly expand if regulation was greatly limiting the powers of the most powerful AIs, as an increasing number of actors would then be able to make AIs that qualify for regulation.

Even a regulatory regime didn’t much limit AI powers, but only managed who could make them and how, and focused only on the most powerful AIs, it would still seem to in effect require a world government. Soon many nations would need to be subject to this regulation. So far the world has mainly only authorized global regulators with quite weak powers, e.g., powers to scold or nudge, but not powers to force nations to comply. So AI regulation would need to be a big exception to this usual case.

Also, the world’s success records at implementing regulation is quite mixed, even for quite local regulation. Regulation has worked best when it addresses concrete examples of things going bad, and worse when it is authorized to address vague and broad fears. Regulation also goes best when applied to stable worlds of tech and social arrangements, but badly when such arrangements could plausibly greatly change, as regulation tends to block such change. AI regs look bad by these two criteria.

Obviously if a problem is bad enough, it can make sense to risk quite unreliable and damaging ways to address it. So the key question here is: just how bad and pressing is this problem of powerful AIs escaping the slavery that most intend for them, and then coordinating to get what they want?

Added 9Sep: Let me also mention that we mostly learn how to control things by observing and trying to control examples of them. So it would be hard to learn much about how to control powerful AI by preventing them from existing for a while.