Author Archives: Eliezer Yudkowsky

On Not Having an Advance Abyssal Plan

By Eliezer Yudkowsky · February 23, 2009 3:20 pm · Comments (39)

"Even though he could foresee the problem then, we can see it equally well now.  Therefore, if he could foresee the solution then, we should be able to see it now.  After all, Seldon was not a magician.  There are no trick methods of escaping a dilemma that he can see and we can't."
        — Salvor Hardin

Years ago at the Singularity Institute, the Board was entertaining a proposal to expand somewhat.  I wasn't sure our funding was able to support the expansion, so I insisted that – if we started running out of money – we decide in advance who got fired and what got shut down, in what order.

Even over the electronic aether, you could hear the uncomfortable silence.

"Why can't we decide that at the time, if the worst happens?" they said, or something along those lines.

"For the same reason that when you're buying a stock you think will go up, you decide how far it has to decline before it means you were wrong," I said, or something along those lines; this being far back enough in time that I would still have used stock-trading in a rationality example.  "If we can make that decision during a crisis, we ought to be able to make it now.  And if I can't trust that we can make this decision in a crisis, I can't trust this to go forward."

People are really, really reluctant to plan in advance for the abyss.  But what good reason is there not to?  How can you be worse off from knowing in advance what you'll do in the worse cases?

I have been trying fairly hard to keep my mouth shut about the current economic crisis.  But still –

Why didn't various governments create and publish a plan for what they would do in the event of various forms of financial collapse, before it actually happened?

Continue reading "On Not Having an Advance Abyssal Plan" »

GD Star Rating
a WordPress rating system

Fairness vs. Goodness

By Eliezer Yudkowsky · February 22, 2009 3:22 pm · Comments (16)

It seems that back when the Prisoner's Dilemma was still being worked out, Merrill Flood and Melvin Drescher tried a 100-fold iterative PD on two smart but unprepared subjects, Armen Alchian of UCLA and John D. Williams of RAND.

The kicker being that the payoff matrix was asymmetrical, with dual cooperation awarding JW twice as many points as AA:

(AA, JW) JW: D JW: C
AA: D (0, 0.5) (1, -1)
AA: C (-1, 2) (0.5, 1)

The resulting 100 iterations, with a log of comments written by both players, make for fascinating reading.

JW spots the possibilities of cooperation right away, while AA is slower to catch on.

But once AA does catch on to the possibilities of cooperation, AA goes on throwing in an occasional D… because AA thinks the natural meeting point for cooperation is a fair outcome, where both players get around the same number of total points.

JW goes on trying to enforce (C, C) – the option that maximizes total utility for both players – by punishing AA's attempts at defection.  JW's log shows comments like "He's crazy.  I'll teach him the hard way."

Meanwhile, AA's log shows comments such as "He won't share.  He'll punish me for trying!"

Continue reading "Fairness vs. Goodness" »

GD Star Rating
a WordPress rating system

Rationality Quotes 27

By Eliezer Yudkowsky · February 21, 2009 8:55 pm · Comments (8)

"Believing this statement will make you happier."
        — Ryan Lortie

"Make changes based on your strongest opportunities, not your most convenient ones."
        — MegaTokyo

"The mind is a cruel, lying, unreliable bastard that can't be trusted with even an ounce of responsibility.  If you were dating the mind, all your friends would take you aside, and tell you that you can really do better, and being alone isn't all that bad, anyway.  If you hired the mind as a babysitter, you would come home to find all but one of your children in critical condition, and the remaining one crowned 'King of the Pit'."
        — Lore Sjoberg

"Getting bored is a non-trivial cerebral transformation that doubtlessly took many millions of years for nature to perfect."
        — Lee Corbin

"The views expressed here do not necessarily represent the unanimous view of all parts of my mind."
        — Malcolm McMahon

Continue reading "Rationality Quotes 27" »

GD Star Rating
a WordPress rating system

Wise Pretensions v.0

By Eliezer Yudkowsky · February 20, 2009 12:02 pm · Comments (22)

Followup to:  Pretending to be Wise

For comparison purposes, here's an essay with similar content to yesterday's "Pretending to be Wise", which I wrote in 2006 in a completely different style, edited down slightly (content has been deleted but not added).  Note that the 2006 concept of "pretending to be Wise" hasn't been narrowed down as much compared to the 2009 version; also when I wrote it, I was in more urgent need of persuasive force.

I thought it would be an interesting data point to check whether this essay seems more convincing than yesterday's, following Robin's injuction "to avoid emotion, color, flash, stories, vagueness, repetition, rambling, and even eloquence" – this seems like rather the sort of thing he might have had in mind.

And conversely the stylistic change also seems like the sort of thing Orwell might have had in mind, when Politics and the English Language compared:  "I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all."  Versus:  "Objective considerations of contemporary phenomena compel the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account."  That would be the other side of it.

At any rate, here goes Eliezer2006…

I do not fit the stereotype of the Wise. I am not Gandalf, Ged, or Gandhi. I do not sit amidst my quiet garden, staring deeply into the truths engraved in a flower or a drop of dew; speaking courteously to all who come before me, and answering them gently regardless of how they speak to me.

If I tried to look Wise, and succeeded, I would receive more respect from my fellows. But there would be a price.

To pretend to be Wise means that you must always appear to give people the benefit of the doubt. Thus people will admire you for your courtesy. But this is not always true.

To pretend to be Wise, you must always pretend that both sides have merit, and solemnly refuse to judge between them. For if you took one side or another, why then, you would no longer be one of the aloof Wise, but merely another partisan, on a level with all the other mere bickerers.

As one of the Wise, you are omnipotent on the condition that you never exercise your power. Otherwise people would start thinking that you were no better than they; and they would no longer hold you in awe.

Continue reading "Wise Pretensions v.0" »

GD Star Rating
a WordPress rating system

Pretending to be Wise

By Eliezer Yudkowsky · February 19, 2009 5:30 pm · Comments (43)

Followup to:  Against Maturity

"The hottest place in Hell is reserved for those who in time of crisis remain neutral."
        — Dante Alighieri, famous hell expert John F. Kennedy, misquoter

A special case of adulthood-signaling worth singling out, is the display of neutrality or suspended judgment, in order to signal maturity, wisdom, impartiality, or just a superior vantage point.

An example would be the case discussed yesterday of my parents, who respond to theological questions like "Why does ancient Egypt, which had good records on many other matters, lack any records of Jews having ever been there?" with "Oh, when I was your age, I also used to ask that sort of question, but now I've grown out of it."

Another example would be the principal who, faced with two children who were caught fighting on the playground, sternly says:  "It doesn't matter who started the fight, it only matters who ends it."  Of course it matters who started the fight.  The principal may not have access to good information about this critical fact, but if so, he should say so, not dismiss the importance of who threw the first punch.  Let a parent try punching the principal, and we'll see how far "It doesn't matter who started it" gets in front of a judge.  But to adults it is just inconvenient that children fight, and it matters not at all to their convenience which child started it, it is only convenient that the fight end as rapidly as possible.

A similar dynamic, I believe, governs the occasions in international diplomacy where Great Powers sternly tell smaller groups to stop that fighting right now.  It doesn't matter to the Great Power who started it – who provoked, or who responded disproportionately to provocation – because the Great Power's ongoing inconvenience is only a function of the ongoing conflict.  Oh, can't Israel and Hamas just get along?

This I call "pretending to be Wise".  Of course there are many ways to try and signal wisdom.  But trying to signal wisdom by refusing to make guesses – refusing to sum up evidence – refusing to pass judgment – refusing to take sides – staying above the fray and looking down with a lofty and condescending gaze – which is to say, signaling wisdom by saying and doing nothing – well, that I find particularly pretentious.

Continue reading "Pretending to be Wise" »

GD Star Rating
a WordPress rating system

Against Maturity

By Eliezer Yudkowsky · February 18, 2009 6:34 pm · Comments (36)

I remember the moment of my first break with Judaism.  It was in kindergarten, when I was being forced to memorize and recite my first prayer.  It was in Hebrew.  We were given a transliteration, but not a translation.  I asked what the prayer meant.  I was told that I didn't need to know – so long as I prayed in Hebrew, it would work even if I didn't understand the words.  (Any resemblance to follies inveighed against in my writings is not coincidental.)

Of course I didn't accept this, since it was blatantly stupid, and I figured that God had to be at least as smart as I was.  So when I got home, I asked my parents, and they didn't bother arguing with me.  They just said, "You're too young to argue with; we're older and wiser; adults know best; you'll understand when you're older."

They were right about that last part, anyway.

Of course there were plenty of places my parents really did know better, even in the realms of abstract reasoning.  They were doctorate-bearing folks and not stupid.  I remember, at age nine or something silly like that, showing my father a diagram full of filled circles and trying to convince him that the indeterminacy of particle collisions was because they had a fourth-dimensional cross-section and they were bumping or failing to bump in the fourth dimension.

My father shot me down flat.  (Without making the slightest effort to humor me or encourage me.  This seems to have worked out just fine.  He did buy me books, though.)

But he didn't just say, "You'll understand when you're older."  He said that physics was math and couldn't even be talked about without math.  He talked about how everyone he met tried to invent their own theory of physics and how annoying this was.  He may even have talked about the futility of "providing a mechanism", though I'm not actually sure if I originally got that off him or Baez.

You see the pattern developing here.  "Adulthood" was what my parents appealed to when they couldn't verbalize any object-level justification.  They had doctorates and were smart; if there was a good reason, they usually would at least try to explain it to me.  And it gets worse…

Continue reading "Against Maturity" »

GD Star Rating
a WordPress rating system

Good Idealistic Books are Rare

By Eliezer Yudkowsky · February 17, 2009 1:41 pm · Comments (20)

Saith Robin in "Seeking a Cynic's Library":

Cynicism and Idealism are a classic yin and yang, a contradictory pair where we all seem to need both sides…

Books on education, medicine, government, charity, religion, technology, travel, relationships, etc. mostly present relatively idealistic views, though of course no view is entirely one way or the other.  So one reason the young tend to be idealistic is that most reading material they can easily find and understand is idealistic. 

My impression of this differs somewhat from Robin's (what a surprise).

I think that what we see in most books of the class Robin describes, are official views.  These official views may leave out many unpleasant elements of the story.  But because officialism also tries to signal authority and maturity, it's hardly likely to permit itself any real hope or enthusiasm.  Perhaps an obligatory if formal nod in the direction of some popular good cause, because this is expected of officialdom.  But this is hardly an idealistic voice.

What does a full-blown nonfictional idealism look like?  Some examples that I remember from my own youth:

  • Jerry Pournelle's A Step Farther Out, an idealistic view of space travel and more general technological advancement, and the possibility of rising standards of living as opposed to Ehrlichian gloomsaying.
  • Brown, Keating, Mellinger, Post, Smith, and Tudor's The Incredible Bread Machine, my childhood introduction to traditional capitalist values.
  • Eric Drexler's Engines of Creation (and to a lesser extent Ed Regis's Great Mambo Chicken), my introduction to transhumanism.
  • Richard Feynman's Surely You're Joking, Mr. Feynman (for traditional rationalist values).

Supposing you wanted your child to grow up an idealist – what nonfiction books like these could you find to give them?  I don't find it easy to think of many – most nonfiction books are not like this.

Continue reading "Good Idealistic Books are Rare" »

GD Star Rating
a WordPress rating system

Cynical About Cynicism

By Eliezer Yudkowsky · February 16, 2009 7:49 pm · Comments (15)

I'm cynical about cynicism.  I don't believe that most cynicism is really about knowing better.  When I see someone being cynical, my first thought is that they're trying to show off their sophistication and assert superiority over the naive.  As opposed to, say, sharing their uncommon insight about not-widely-understood flaws in human nature.

There are two obvious exceptions to this rule.  One is if the speaker has something serious and realistic to say about how to improve matters.  Claiming that problems can be fixed will instantly lose you all your world-weary street cred and mark you as another starry-eyed idealistic fool.  (Conversely, any "solution" that manages not to disrupt the general atmosphere of doom, does not make me less skeptical:  "Humans are evil creatures who slaughter and destroy, but eventually we'll die out from poisoning the environment, so it's all to the good, really.")

No, not every problem is solvable.  But by and large, if someone achieves uncommon insight into darkness – if they know more than I do about human nature and its flaws – then it's not unreasonable to expect that they might have a suggestion or two to make about remedy, patching, or minor deflection.  If, you know, the problem is one that they really would prefer solved, rather than gloom being milked for a feeling of superiority to the naive herd.

The other obvious exception is for science that has something to say about human nature.  A testable hypothesis is a testable hypothesis and the thing to do with it is test it.  Though here one must be very careful not to go beyond the letter of the experiment for the sake of signaling hard-headed realism:

Continue reading "Cynical About Cynicism" »

GD Star Rating
a WordPress rating system

An African Folktale

By Eliezer Yudkowsky · February 15, 2009 8:00 pm · Comments (44)

This is a folktale of the Hausa, a farming culture of around 30 million people, located primarily in Nigeria and Niger but with other communities scattered around Africa.  I find the different cultural assumptions revealed to be… attention-catching; you wouldn't find a tale like this in Aesop.  From Hausa Tales and Traditions by Frank Edgar and Neil Skinner; HT Robert Greene.

The Farmer, the Snake and the Heron

    There was once a man hoeing away on his farm, when along came some people chasing a snake, meaning to kill it.  And the snake came up to the farmer.
    Says the snake "Farmer, please hide me."  "Where shall I hide you?" said the farmer, and the snake said "All I ask is that you save my life."  The farmer couldn't think where to put the snake, and at last bent down and opened his anus, and the snake entered.
    Presently the snake's pursuers arrived and said to the farmer "Hey, there!  Where's the snake we were chasing and intend to kill?  As we followed him, he came in your direction."  Says the farmer "I haven't seen him."  And the people went back again.
    Then the farmer said to the snake "Righto – come out now.  They've gone."  "Oh no" said the snake, "I've got me a home."  And there was the farmer, with his stomach all swollen, for all the world like a pregnant woman!

Continue reading "An African Folktale" »

GD Star Rating
a WordPress rating system

Rationality Quotes 26

By Eliezer Yudkowsky · February 14, 2009 11:14 am · Comments (15)

"Poets, philosophers, acidheads, salesmen: everybody wants to know, 'What is Reality?'  Some say it's a vast Unknowable so astounding and raw and naked that it grips the human mind and shakes it like a puppy shakes a rag doll.  A lot of good that does us."
        — The Book of the SubGenius

"When they discovered that reality was more complicated than they thought, they just swept the complexity under a carpet of epicycles.  That is, they created unnecessary complexity.  This is an important point.  The universe is complex, but it's usefully complex."
        — Larry Wall

"I can't imagine a more complete and precise answer to the question 'for what reason…?' than 'none'.  The fact that you don't like the answer is your problem, not the universe's."
        — Lee Daniel Crocker

"In the end they all moved in fantasies and not in the daily tide of their seemingly useless lives.  Souls forever lost in the terrifying freedom of their existence."
        — Shinji and Warhammer40k

"Thus the freer the judgement of a man is in regard to a definite issue, with so much greater necessity will the substance of this judgement be determined."
        — Friedrich Engels, Anti-Dühring

Continue reading "Rationality Quotes 26" »

GD Star Rating
a WordPress rating system