Category Archives: Naturalism

Nonsentient Optimizers

By Eliezer Yudkowsky · December 26, 2008 9:32 pm · Comments (38)

Followup to: Nonperson Predicates, Possibility and Could-ness

    "All our ships are sentient.  You could certainly try telling a ship what to do… but I don't think you'd get very far."
    "Your ships think they're sentient!" Hamin chuckled.
    "A common delusion shared by some of our human citizens."
            — Player of Games, Iain M. Banks

Yesterday, I suggested that, when an AI is trying to build a model of an environment that includes human beings, we want to avoid the AI constructing detailed models that are themselves people.  And that, to this end, we would like to know what is or isn't a person – or at least have a predicate that returns 1 for all people and could return 0 or 1 for anything that isn't a person, so that, if the predicate returns 0, we know we have a definite nonperson on our hands.

And as long as you're going to solve that problem anyway, why not apply the same knowledge to create a Very Powerful Optimization Process which is also definitely not a person?

"What?  That's impossible!"

How do you know?  Have you solved the sacred mysteries of consciousness and existence?

"Um – okay, look, putting aside the obvious objection that any sufficiently powerful intelligence will be able to model itself -"

Lob's Sentence contains an exact recipe for a copy of itself, including the recipe for the recipe; it has a perfect self-model.  Does that make it sentient?

"Putting that aside – to create a powerful AI and make it not sentient – I mean, why would you want to?"

Several reasons.  Picking the simplest to explain first – I'm not ready to be a father.

Continue reading "Nonsentient Optimizers" »

GD Star Rating
loading...

Nonperson Predicates

By Eliezer Yudkowsky · December 26, 2008 8:47 pm · Comments (45)

Followup to:  Righting a Wrong Question, Zombies! Zombies?, A Premature Word on AI, On Doing the Impossible

There is a subproblem of Friendly AI which is so scary that I usually don't talk about it, because only a longtime reader of Overcoming Bias would react to it appropriately – that is, by saying, "Wow, that does sound like an interesting problem", instead of finding one of many subtle ways to scream and run away.

This is the problem that if you create an AI and tell it to model the world around it, it may form models of people that are people themselves.  Not necessarily the same person, but people nonetheless.

If you look up at the night sky, and see the tiny dots of light that move over days and weeks – planētoi, the Greeks called them, "wanderers" – and you try to predict the movements of those planet-dots as best you can…

Historically, humans went through a journey as long and as wandering as the planets themselves, to find an accurate model.  In the beginning, the models were things of cycles and epicycles, not much resembling the true Solar System.

But eventually we found laws of gravity, and finally built models – even if they were just on paper – that were extremely accurate so that Neptune could be deduced by looking at the unexplained perturbation of Uranus from its expected orbit.  This required moment-by-moment modeling of where a simplified version of Uranus would be, and the other known planets.  Simulation, not just abstraction.  Prediction through simplified-yet-still-detailed pointwise similarity.

Suppose you have an AI that is around human beings.  And like any Bayesian trying to explain its enivornment, the AI goes in quest of highly accurate models that predict what it sees of humans.

Models that predict/explain why people do the things they do, say the things they say, want the things they want, think the things they think, and even why people talk about "the mystery of subjective experience".

The model that most precisely predicts these facts, may well be a 'simulation' detailed enough to be a person in its own right.

Continue reading "Nonperson Predicates" »

GD Star Rating
loading...

Lawful Creativity

By Eliezer Yudkowsky · November 8, 2008 2:54 pm · Comments (26)

Previously in Series:  Recognizing Intelligence

Creativity, we’ve all been told, is about Jumping Out Of The System, as Hofstadter calls it (JOOTSing for short).  Questioned assumptions, violated expectations.

Fire is dangerous: the rule of fire is to run away from it.  What must have gone through the mind of the first hominid to domesticate fire?  The rule of milk is that it spoils quickly and then you can’t drink it – who first turned milk into cheese?  The rule of computers is that they’re made with vacuum tubes, fill a room and are so expensive that only corporations can own them.  Wasn’t the transistor a surprise…

Who, then, could put laws on creativity?  Who could bound it, who could circumscribe it, even with a concept boundary that distinguishes "creativity" from "not creativity"?  No matter what system you try to lay down, mightn’t a more clever person JOOTS right out of it?  If you say "This, this, and this is ‘creative’" aren’t you just making up the sort of rule that creative minds love to violate?

Why, look at all the rules that smart people have violated throughout history, to the enormous profit of humanity.  Indeed, the most amazing acts of creativity are those that violate the rules that we would least expect to be violated.

Is there not even creativity on the level of how to think?  Wasn’t the invention of Science a creative act that violated old beliefs about rationality?  Who, then, can lay down a law of creativity?

But there is one law of creativity which cannot be violated…

Continue reading "Lawful Creativity" »

GD Star Rating
loading...

Mundane Magic

By Eliezer Yudkowsky · October 31, 2008 12:00 pm · Comments (29)

Followup to:  Joy in the Merely Real, Joy in Discovery, If You Demand Magic, Magic Won’t Help

As you may recall from some months earlier, I think that part of the rationalist ethos is binding yourself emotionally to an absolutely lawful reductionistic universe – a universe containing no ontologically basic mental things such as souls or magic – and pouring all your hope and all your care into that merely real universe and its possibilities, without disappointment.

There’s an old trick for combating dukkha where you make a list of things you’re grateful for, like a roof over your head.

So why not make a list of abilities you have that would be amazingly cool if they were magic, or if only a few chosen individuals had them?

For example, suppose that instead of one eye, you possessed a magical second eye embedded in your forehead.  And this second eye enabled you to see into the third dimension – so that you could somehow tell how far away things were – where an ordinary eye would see only a two-dimensional shadow of the true world.  Only the possessors of this ability can accurately aim the legendary distance-weapons that kill at ranges far beyond a sword, or use to their fullest potential the shells of ultrafast machinery called "cars".

"Binocular vision" would be too light a term for this ability.  We’ll only appreciate it once it has a properly impressive name, like Mystic Eyes of Depth Perception.

So here’s a list of some of my favorite magical powers:

Continue reading "Mundane Magic" »

GD Star Rating
loading...

Entangled Truths, Contagious Lies

By Eliezer Yudkowsky · October 15, 2008 7:39 pm · Comments (20)

"One of your very early philosophers came to the conclusion that a fully competent mind, from a study of one fact or artifact belonging to any given universe, could construct or visualize that universe, from the instant of its creation to its ultimate end…"
        — First Lensman

"If any one of you will concentrate upon one single fact, or small object, such as a pebble or the seed of a plant or other creature, for as short a period of time as one hundred of your years, you will begin to perceive its truth."
        — Gray Lensman

I am reasonably sure that a single pebble, taken from a beach of our own Earth, does not specify the continents and countries, politics and people of this Earth.  Other planets in space and time, other Everett branches, would generate the same pebble.  On the other hand, the identity of a single pebble would seem to include our laws of physics.  In that sense the entirety of our Universe – all the Everett branches – would be implied by the pebble.  (If, as seems likely, there are no truly free variables.)

So a single pebble probably does not imply our whole Earth.  But a single pebble implies a very great deal.  From the study of that single pebble you could see the laws of physics and all they imply.  Thinking about those laws of physics, you can see that planets will form, and you can guess that the pebble came from such a planet.  The internal crystals and molecular formations of the pebble formed under gravity, which tells you something about the planet’s mass; the mix of elements in the pebble tells you something about the planet’s formation.

I am not a geologist, so I don’t know to which mysteries geologists are privy.  But I find it very easy to imagine showing a geologist a pebble, and saying, "This pebble came from a beach at Half Moon Bay", and the geologist immediately says, "I’m confused" or even "You liar".  Maybe it’s the wrong kind of rock, or the pebble isn’t worn enough to be from a beach – I don’t know pebbles well enough to guess the linkages and signatures by which I might be caught, which is the point.

Continue reading "Entangled Truths, Contagious Lies" »

GD Star Rating
loading...

Beyond the Reach of God

By Eliezer Yudkowsky · October 4, 2008 11:42 am · Comments (160)

Followup to:  The Magnitude of His Own Folly

Today’s post is a tad gloomier than usual, as I measure such things.  It deals with a thought experiment I invented to smash my own optimism, after I realized that optimism had misled me.  Those readers sympathetic to arguments like, "It’s important to keep our biases because they help us stay happy," should consider not reading.  (Unless they have something to protect, including their own life.)

So!  Looking back on the magnitude of my own folly, I realized that at the root of it had been a disbelief in the Future’s vulnerability – a reluctance to accept that things could really turn out wrong.  Not as the result of any explicit propositional verbal belief.  More like something inside that persisted in believing, even in the face of adversity, that everything would be all right in the end.

Some would account this a virtue (zettai daijobu da yo), and others would say that it’s a thing necessary for mental health.

But we don’t live in that world.  We live in the world beyond the reach of God.

Continue reading "Beyond the Reach of God" »

GD Star Rating
loading...

My Naturalistic Awakening

By Eliezer Yudkowsky · September 25, 2008 2:58 am · Comments (42)

Followup to:  Fighting a Rearguard Action Against the Truth

In yesterday’s episode, Eliezer2001 is fighting a rearguard action against the truth.  Only gradually shifting his beliefs, admitting an increasing probability in a different scenario, but never saying outright, "I was wrong before."  He repairs his strategies as they are challenged, finding new justifications for just the same plan he pursued before.

(Of which it is therefore said:  "Beware lest you fight a rearguard retreat against the evidence, grudgingly conceding each foot of ground only when forced, feeling cheated.  Surrender to the truth as quickly as you can.  Do this the instant you realize what you are resisting; the instant you can see from which quarter the winds of evidence are blowing against you.")

Memory fades, and I can hardly bear to look back upon those times – no, seriously, I can’t stand reading my old writing.  I’ve already been corrected once in my recollections, by those who were present.  And so, though I remember the important events, I’m not really sure what order they happened in, let alone what year.

But if I had to pick a moment when my folly broke, I would pick the moment when I first comprehended, in full generality, the notion of an optimization process.  That was the point at which I first looked back and said, "I’ve been a fool."

Continue reading "My Naturalistic Awakening" »

GD Star Rating
loading...

The Sheer Folly of Callow Youth

By Eliezer Yudkowsky · September 18, 2008 9:30 pm · Comments (13)

Followup to:  My Childhood Death Spiral, My Best and Worst Mistake, A Prodigy of Refutation

"There speaks the sheer folly of callow youth; the rashness of an ignorance so abysmal as to be possible only to one of your ephemeral race…"
        — Gharlane of Eddore

Once upon a time, years ago, I propounded a mysterious answer to a mysterious question – as I’ve hinted on several occasions.  The mysterious question to which I propounded a mysterious answer was not, however, consciousness – or rather, not only consciousness.  No, the more embarrassing error was that I took a mysterious view of morality.

I held off on discussing that until now, after the series on metaethics, because I wanted it to be clear that Eliezer1997 had gotten it wrong.

When we last left off, Eliezer1997, not satisfied with arguing in an intuitive sense that superintelligence would be moral, was setting out to argue inescapably that creating superintelligence was the right thing to do.

Well (said Eliezer1997) let’s begin by asking the question:  Does life have, in fact, any meaning?

Continue reading "The Sheer Folly of Callow Youth" »

GD Star Rating
loading...

Psychic Powers

By Eliezer Yudkowsky · September 12, 2008 3:28 pm · Comments (34)

Followup to: Excluding the Supernatural

Yesterday, I wrote:

If the "boring view" of reality is correct, then you can never predict anything irreducible because you are reducible.  You can never get Bayesian confirmation for a hypothesis of irreducibility, because any prediction you can make is, therefore, something that could also be predicted by a reducible thing, namely your brain.

Benja Fallenstein commented:

I think that while you can in this case never devise an empirical test whose outcome could logically prove irreducibility, there is no clear reason to believe that you cannot devise a test whose counterfactual outcome in an irreducible world would make irreducibility subjectively much more probable (given an Occamian prior).

Without getting into reducibility/irreducibility, consider the scenario that the physical universe makes it possible to build a hypercomputer — that performs operations on arbitrary real numbers, for example — but that our brains do not actually make use of this: they can be simulated perfectly well by an ordinary Turing machine, thank you very much…

Well, that’s a very intelligent argument, Benja Fallenstein.  But I have a crushing reply to your argument, such that, once I deliver it, you will at once give up further debate with me on this particular point:

Continue reading "Psychic Powers" »

GD Star Rating
loading...

Excluding the Supernatural

By Eliezer Yudkowsky · September 11, 2008 8:12 pm · Comments (47)

Followup to:  Reductionism, Anthropomorphic Optimism

Occasionally, you hear someone claiming that creationism should not be taught in schools, especially not as a competing hypothesis to evolution, because creationism is a priori and automatically excluded from scientific consideration, in that it invokes the "supernatural".

So… is the idea here, that creationism could be true, but even if it were true, you wouldn’t be allowed to teach it in science class, because science is only about "natural" things?

It seems clear enough that this notion stems from the desire to avoid a confrontation between science and religion.  You don’t want to come right out and say that science doesn’t teach Religious Claim X because X has been tested by the scientific method and found false.  So instead, you can… um… claim that science is excluding hypothesis X a priori.  That way you don’t have to discuss how experiment has falsified X a posteriori.

Of course this plays right into the creationist claim that Intelligent Design isn’t getting a fair shake from science – that science has prejudged the issue in favor of atheism, regardless of the evidence.  If science excluded Intelligent Design a priori, this would be a justified complaint!

But let’s back up a moment.  The one comes to you and says:  "Intelligent Design is excluded from being science a priori, because it is ‘supernatural’, and science only deals in ‘natural’ explanations."

What exactly do they mean, "supernatural"?  Is any explanation invented by someone with the last name "Cohen" a supernatural one?  If we’re going to summarily kick a set of hypotheses out of science, what is it that we’re supposed to exclude?

By far the best definition I’ve ever heard of the supernatural is Richard Carrier’s:  A "supernatural" explanation appeals to ontologically basic mental things, mental entities that cannot be reduced to nonmental entities.

Continue reading "Excluding the Supernatural" »

GD Star Rating
loading...