A few years ago I recorded 13 convos w/ folks worried about AI risk, collected at YouTube. I usually asked people why they wanted to treat AI descendants differently from bio-squishy ones. Most accepted future human-level AIs as descendants; we are making them in our image, and they will be more like us than are AIs made by aliens. Most were also okay with bio-squishy descendants usually getting more powerful than, winning conflicts with, and changing values relative to, prior generations. Even so, most still objected to future AI descendants having a similar relation to us.
They gave three main arguments. First, some doubted that AIs would be conscious, due their being made out of different physical materials. Making them unworthy descendants, regardless of their behaviors or values. Second, some expected AI value changes to eventually get much larger than future bio-squishy value changes, due to some human values being encoded in slow-to-change DNA. I think this greatly misunderstands human nature - most values we cherish today were given to us by culture, not DNA. Culture is how we now pass on our values to both AI and other descendants. Also, such folks should object less to AIs with a less-pliable value core.
The third main argument given was that the speed of social change would be too fast with AIs. Folks saying this accept that rates of social change today are now much faster than they were in our past, and greatly approve of this speedup. They also didn’t mind human values changing greatly over the coming generations, and even hoped to greatly extend their lifetimes to overlap more with later generations who will have very different values. However, they greatly objected to the possibility that AIs might have shorter generation times, and thus induce social change faster than we see today.
Such folks see themselves as having a deep right to social change that is no faster than the rates to which they have become familiar; anything faster is an existential risk. So most endorse strong efforts now to create a worldwide governance process that at first controls just the top AI labs, but soon controls all computer systems above set capability levels. As the cost of computing falls fast, this would soon require a fast expanding scope of control over all the computing power in the world.
However, an X poll suggests that only ~8% of people would mind a tech that makes rates of social change ten times faster for bio-squishy descendants, via creating human adults in two instead of 20 years. Thus this third argument seems to me like a mask for one like first two: claims that bio-squishy humans are just intrinsically special, compared to anything that metal could make.