Future Incompetence

The book Human Enhancement is finally out.  My chapter is second to last, just after a thoughtful one by Daniel Wikler:

It is often observed that mildly or even moderately retarded people do not seem dull to themselves as long as they stay on the farm (rather: certain farms), but become so immediately when they move to the city.  Here the relative difference between a dull or not-very-bright minority and the majority who are just below average, or better, becomes important, and as that majority arranges society to suit themselves, their less-bright peers become incompetent. …

Those who are rendered incompetent in this manner need supervision, and in order to protect them in that now-dangerous environment, their rights are taken away.  Humane regimes strive to protect as uch of their range of free choice as possible, consistent with the need to protect them from serious or irremediable harm (and to protect others), but there is no supposition that everyone has a natural, inalienable right to self-determination  that would rule out all configurations of the social and physical environment that are disadvantageous to the less-talented. …

What, then, would be the effect of selective enhancement of intellectual capacity – that is, enhancement of some, but not all – for the social and political world that we "normals" would inhabit?  Would it erode the foundations of egalitarianism, undermining the claims of many who now hold title ans citizens to that equal status?  Would those made or engineered to be born smart be within their rights to deprive the rest of our rights, presumably with a humanitarian intent?  In a word: yes. … 

Should we be eternally vigilant and suspicious of people who appoint themselves "guardians", profess humanitarian motives, and then take over our lives?  Or do the shoes just hurt because they would be on our feet?

This is a great test case for paternalists; if you feel that your superior minds justify ruling the lives of others, would do you accept having your life ruled by future folk with greatly enhanced minds?

GD Star Rating
Tagged as: ,
Trackback URL:
  • Presumably anyone who supports the FAI position implicitly supports the idea of a smart mind supervising lesser minds for their own benefit, including the case where the lesser mind is them.

    I for one welcome our new FAI overlord, because at least that way I get to not die, be happy, be more intelligent, and generally live a more fulfilling life by my own standards.

  • self determination is just one preference among many. some people hold it higher than all others (or at least claim to) and some are willing to trade in many situations when it allows them to fulfill other preferences. (see: all of society)

    preferences do change depending on how much control over a situation a person has. it shouldn’t be surprising that people change their tune when they find out that they won’t be calling the shots anymore.

  • Peter Twieg

    I’d imagine that a lot of paternalists could avoid the problem by simply saying that it’s in no one’s Enlightened Interest to make themselves smarter than the paternalists.

    But in any case, I’m not sure how meaningful this argument is because I don’t think we abridge the rights of the mildly retarded to too great an extent in most circumstances. The stronger point is, though, that there’s presumably an upper bound on rationality (if not computational power), and creating super-intelligent individuals wouldn’t necessarily increase their rational abilities, just their processing speed. And since presumably paternalistic rights-infringement occurs because of a lack of rationality, not processing speed… so as we asymptotically approximate “perfect rationality” via technological improvements, I think we’d reach a point where transhumans of generation N+1 do not have a sufficient advantage in terms of rationality over transhumans of generation N to justify a paternalistic abridgment of rights. The major reason against a paternalistic regime isn’t simply incentive issues (will the paternalists really be benevolent?), but information issues – will the paternalists really know what’s in your interests, even if they have a cognitive advantage over you? I doubt that the cognition gap (which favors paternalism) will close as quickly as the information gap (which does not), but I could be wrong.

  • I for one welcome our new FAI overlord, because at least that way I get to not die, be happy, be more intelligent, and generally live a more fulfilling life by my own standards.

    Heh heh heh 🙂

    If your FAI inherits any of our values, it may farm us in henhouses like we do chickens. 🙂

  • Fenn

    Man, I hope it inherits enough of our values to at least farm us in people houses.

  • Grant

    The paternalists’ moral hypocrisy doesn’t seem like evidence that their arguments are incorrect. Nearly everyone is self-serving and morally hypocritical, and yet many people put forward arguments which benefit others.

  • I think we’re being incompetently paternalistic by destroying farms and replacing them with asylums. (See also the parts of Fun Theory that deal with self-determination.)

  • Roko

    @eli: destroying the farms and replacing them with asylums? I don’t get that…

  • Gwailo

    Didn’t Philip K. Dick have something to say about this @ 40 years ago?

  • Sideways

    You don’t need to take developmental disabilities into account to see that some people are already more competent to live in society than others. Intelligence isn’t the only factor–sociability, wealth, and membership in high-status groups (racial, religious, etc.) result in the familiar disparities.

    Eliezer, assuming that by “farms” you mean situations in general where disabled people are empowered, there is no agenda to “destroy farms and replace them with asylums.” The human world is just becoming more complicated, harder to navigate, and in general less “farm-like.” Homes for adults with developmental disabilities are legally and ethically bound to empower their residents to live in the world, not to isolate them as the term ‘asylum’ implies.

    Based on this excerpt, Wikler’s concept of a “natural, inalienable right” to live in a simple society sounds like a straw man argument. Adults with disabilities can and do live well in our complicated world, if they have the support they need. They vote, marry, have careers, give back to their communities, and so on. They do not want society to regress to accommodate them. I hope that in a world of supermen, we “normals” could accept our limitations just as gracefully.

  • ad

    The human world is just becoming more complicated, harder to navigate, and in general less “farm-like.”

    Is it? If I were dropped off without warning somewhere in e.g. the 18th century, I might end up a lot hungrier than someone from the 18th century brought forwards 200 years.

  • Sideways

    ad: fun to speculate about, but not the right question to ask given the topic. The right question is: assuming no support services, would someone with mild to moderate retardation be more empowered in NYC in the 18th century, or in NYC today? What about an 18th century farm versus a modern farm? Wikler has already given us the answer to these questions, though, so they aren’t all that fun to ask.

    Complicating the issue is that until fairly recently “support services” were in place that did more harm than good, like Bedlam-houses, electroshock, and lobotomy. Hopefully, future superintelligent overlords wouldn’t treat “normals” that way.

  • haig

    Soft (libertarian) paternalism has always been my ideology of choice when I must be governed by intelligences/powers stronger than my own.

  • ad

    Sideways: The way I read that extract, Wikler was comparing the modern city to a modern farm, which is not the same thing as comparing modern NYC to 18th century NYC.

    The claim that intelligence is a bigger advantage now than it was in the past, seems to me to be different from Wiklers claim that intelligence is a bigger advantage in NYC than it is on a farm. (It is also different from the claim that intelligence will be a bigger advantage in the future after human enhancement than it is now.)

  • Kios

    “They do not want society to regress to accommodate them.” -Sideways

    Scented candles come with warning labels about ingestion, vacuums with warning about genital insertion, and cups of coffee with warnings about the contents being hot. The original lawsuits that prompted these liability controls are predicated on the same problems that keep classrooms learning at the pace of the least-invested or apt students in the room, and television, movie, and advertisement production insipid and puerile enough to bore an intelligent eight year old. Regardless of desire the effects are obvious.

  • Bjarne


  • Pingback: Cavemen debating society, Zombies created by crowds, Liquidating some gov, Making worse decisions that feel better, Assortative-mating created classes « Mike Kenny()