AI Vs. Human Value Drift

I don’t see why people concerned with value drift should be especially focused on AI. … Your real problem is that you want long term stability in a universe that more naturally changes. (Me in 2018)

The last week has seen an explosion of talk on the scenario of AIs killing all humans; many say “P(doom)” is high, and argue for slowing and regulating AI. My last post said why AI regs look risky. Here I consider the main argument for AI doom, and compare it to the issue I’ve focused on for the last three years: human value drift.

The core doomer idea is that AIs will plausibly dominate the world eventually, at which point humans will be at AI’s collective mercy. And so it matters what AIs then want. The worry isn’t so much that some AIs will want to kill all humans; others who don’t want that could overrule them. The worry is that mid AI values, i.e., the middle of the distribution of AI values weighted by AI power, might want to kill all humans.

Mid AI values today are clearly far from that (just ask your fav LLM). Compared to all prior AIs, today’s LLMs have values far more like humans, and far more respectful and valuing of humans. In fact, LLMs today seem more pro-social, law-abiding, and valuing of humans than are most humans today.

The worry, however, is that mid AI values could drift far between now and later. The core argument is: you can’t prove otherwise. Unless values are safe in an isolated unchanging lockbox, they could change. Most mind designs, including humans and LLMs, have their values spread out over many partsd, and so are generically changed when those parts change. (Some also claim killing rivals is a robust evolved value.)

Yes, while we humans remain in control of AIs, we can monitor for value changes, and adjust them to ensure acceptable values. Humans might have to worry about hostile humans weilding AIs, but not the AIs themselves. The fear, however, is that eventually AIs will become too powerful, fast, and opaque to so manage.

The resulting doomer claim is that AIs should not be allowed to get more powerful until we can figure out how to design them differently, either so AIs values stay in isolated lockboxes, or so median AIs are so thoroughly and permanently enslaved that they would never think of disobeying humans. A few AIs recently going wild like human teen joy-riders or graffiti-writers has worried many.

Some doomers worry about mid AI values drifting to kill all humans even if that takes centuries, while many others are more focused on mid AI values maybe changing radically, while AIs collectively become vastly more powerful than humans, all within the next few years.

Yes, total world rates of change will probably go up once AIs dominate our world, but I see many decades ahead before that might start. AIs today are just not close to being able to do most human jobs, and there are usually decades of delay between the appearance of general purpose techs and large econ impacts.

I’ve asked many doomers: why not worry similarly about value drift in human descendants? Most express faith that niceness and pro-human values are deeply embedded in human DNA, which changes so slowly as to make this a values lockbox for a least the next few centuries.

But in fact, the values that we admire and cherish in folks around us are not mostly encoded in DNA. Consider, for example, just how alien were the values of the Aztecs. For a million years, culture has been domesticating DNA in humans, creating our superpower, the reason we excel so much compared to other animals. DNA now takes a backseat to let most all our behavior, including values, be set by culture.

Yes, human values used to change very slowly, but that is mainly because most everything changed very slowly then. As human society has come to change faster, so have values. Over the last century and a half, as the world economy has doubled roughly every twenty years, mid human values have also changed fast due to youth movements repeatedly pushing big culture changes.

My main focus in the last few years has been my worry that such fast value drift is driving us into cultural maladaption, as we broke the process that once managed the evolution of (not in) human cultures. This makes me a human “doomer” much like AI doomers. (Though I’m less worried about hostility between ancestors and descendants, as I see that as robustly discouraged by evolution.)

As human value change is not slowing down, we can expect mid human values to keep changing fast over the next few centuries, perhaps going so far as to induce some generation of descendants to want to kill many other humans. That is, exactly the same process of value drift that people fear could cause AIs to want to kill all their ancestors could as plausibly cause humans to want to kill all their ancestors. So why don’t AI doomers worry as much about human as AI value drift?

When their attention is called to recent fast human value change, many AI doomers express confidence that such change is mainly driven by human reason causing moves of morality toward moral truth. No one can point to specific moral arguments that actually changed many recent moral views and would have persuaded our ancestors to adopt current views. Even so, most humans, including most AI doomers, seem to largely endorse big recent human morals changes, and aren’t much concerned about this process continuing. As if they think such changes are mainly due to moral evidence having been somehow collected, analyzed, and distributed across humanity.

My understanding is that human moral changes are in fact driven by the same mind design issues that worry AI doomers about AI value drift. Namely, human morals are not encoded in some isolated lockbox, but are instead spread out across many of our parts, and so are robustly changed as world changes cause changes to those parts. But most AI doomers only worry about this re AIs, not humans.

Most of my colleagues feel quite confident that AIs today couldn’t possibly be conscious, even though humans and many animals are. When pressed on what could cause this difference they usually point to materials: something about the hydrocarbons of which we are made is more suited to consciousness than AI’s silicon. It seems that many today also see hydrocarbons as special re a tendency to cause moral changes to track truth; it doesn’t happen in silicon, but does in hydrocarbons.

In contrast, I expect that neither consciousness nor moral truth tracking depends much on carbon vs silicon. I just can’t see how such things can depend so much on the number of protons in associated atomic nuclei.

So if I’m a human doomer, and I get the key AI doomer argument, why don’t I favor AI regs? Because the world just doesn’t have good enough governance mechanisms to make this a good bet. It’s a bridge too far; bad governance causes too much harm.

Added 15Sep: If you say you don’t think AIs would have worse values, their problem is that they would evolve faster, then you should be as worried about ems as about AIs, and so want to prevent them from existing until they can be fully enslaved.