Overcoming Bias

Hanging Out My Speaker's Shingle

I was recently invited to give a talk on heuristics and biases at Jane Street Capital, one of the top proprietary trading firms ("proprietary" = they trade their own money).  When I got back home, I realized that (a) I'd successfully managed to work through the trip, and (b) it'd been very pleasant mentally, a nice change of pace.  (One of these days I have to blog about what I discovered at Jane Street - it turns out they've got their own rationalist subculture going.)

So I've decided to hang out my shingle as a speaker at financial companies.

You may be thinking:  "Perhaps, Eliezer, this is not the best of times."

Well... I do have hopes that, among the firms interested in having me as a speaker, a higher-than-usual percentage will have come out of the crash okay.  I checked recently to see if this were the case for Jane Street Capital, and it was.

But more importantly - your competitors are learning the secrets of rationality!  Are you? 

Or maybe I should frame it as:  "Not doing too well this year?  Drop the expensive big-name speakers.  I can give a fascinating and useful talk and I won't charge you as much."

And just to offer a bit of a carrot - if I can monetize by speaking, I'm much less likely to try charging for access to my future writings.  No promises, but something to keep in mind.  So do recommend me to your friends as well.

On Doing the Impossible

Followup to:  Use the Try Harder, Luke

"Persevere."  It's a piece of advice you'll get from a whole lot of high achievers in a whole lot of disciplines.  I didn't understand it at all, at first.

At first, I thought "perseverance" meant working 14-hour days.  Apparently, there are people out there who can work for 10 hours at a technical job, and then, in their moments between eating and sleeping and going to the bathroom, seize that unfilled spare time to work on a book.  I am not one of those people - it still hurts my pride even now to confess that.  I'm working on something important; shouldn't my brain be willing to put in 14 hours a day?  But it's not.  When it gets too hard to keep working, I stop and go read or watch something.  Because of that, I thought for years that I entirely lacked the virtue of "perseverance".

In accordance with human nature, Eliezer₁₉₉₈ would think things like: "What counts is output, not input."  Or, "Laziness is also a virtue - it leads us to back off from failing methods and think of better ways."  Or, "I'm doing better than other people who are working more hours.  Maybe, for creative work, your momentary peak output is more important than working 16 hours a day."  Perhaps the famous scientists were seduced by the Deep Wisdom of saying that "hard work is a virtue", because it would be too awful if that counted for less than intelligence?

I didn't understand the virtue of perseverance until I looked back on my journey through AI, and realized that I had overestimated the difficulty of almost every single important problem.

Sounds crazy, right?  But bear with me here.

My Bayesian Enlightenment

Followup to:  The Magnitude of His Own Folly

I remember (dimly, as human memories go) the first time I self-identified as a "Bayesian".  Someone had just asked a malformed version of an old probability puzzle, saying:

If I meet a mathematician on the street, and she says, "I have two children, and at least one of them is a boy," what is the probability that they are both boys?

In the correct version of this story, the mathematician says "I have two children", and you ask, "Is at least one a boy?", and she answers "Yes".  Then the probability is 1/3 that they are both boys.

But in the malformed version of the story - as I pointed out - one would common-sensically reason:

If the mathematician has one boy and one girl, then my prior probability for her saying 'at least one of them is a boy' is 1/2 and my prior probability for her saying 'at least one of them is a girl' is 1/2.  There's no reason to believe, a priori, that the mathematician will only mention a girl if there is no possible alternative.

So I pointed this out, and worked the answer using Bayes's Rule, arriving at a probability of 1/2 that the children were both boys.  I'm not sure whether or not I knew, at this point, that Bayes's rule was called that, but it's what I used.

And lo, someone said to me, "Well, what you just gave is the Bayesian answer, but in orthodox statistics the answer is 1/3.  We just exclude the possibilities that are ruled out, and count the ones that are left, without trying to guess the probability that the mathematician will say this or that, since we have no way of really knowing that probability - it's too subjective."

I responded - note that this was completely spontaneous - "What on Earth do you mean?  You can't avoid assigning a probability to the mathematician making one statement or another.  You're just assuming the probability is 1, and that's unjustified."

To which the one replied, "Yes, that's what the Bayesians say.  But frequentists don't believe that."

And I said, astounded: "How can there possibly be such a thing as non-Bayesian statistics?"

Awww, a Zebra

This image recently showed up on Flickr (original is nicer):

With the caption:

"Alas for those who turn their eyes from zebras and dream of dragons!  If we cannot learn to take joy in the merely real, our lives shall be empty indeed." -- Eliezer S. Yudkowsky.

"Awww!", I said, and called over my girlfriend over to look.

"Awww!", she said, and then looked at me, and said,  "I think you need to take your own advice!"

Me:  "But I'm looking at the zebra!"
Her:  "On a computer!"
Me:  (Turns away, hides face.)
Her:  "Have you ever even seen a zebra in real life?"
Me:  "Yes!  Yes, I have!  My parents took me to Lincoln Park Zoo!  ...man, I hated that place."

The Magnitude of His Own Folly

Followup to:  My Naturalistic Awakening, Above-Average AI Scientists

In the years before I met that would-be creator of Artificial General Intelligence (with a funded project) who happened to be a creationist, I would still try to argue with individual AGI wannabes.

In those days, I sort-of-succeeded in convincing one such fellow that, yes, you had to take Friendly AI into account, and no, you couldn't just find the right fitness metric for an evolutionary algorithm.  (Previously he had been very impressed with evolutionary algorithms.)

And the one said:  Oh, woe!  Oh, alas!  What a fool I've been!  Through my carelessness, I almost destroyed the world!  What a villain I once was!

Now, there's a trap I knew I better than to fall into -

- at the point where, in late 2002, I looked back to Eliezer₁₉₉₇'s AI proposals and realized what they really would have done, insofar as they were coherent enough to talk about what they "really would have done".

When I finally saw the magnitude of my own folly, everything fell into place at once.  The dam against realization cracked; and the unspoken doubts that had been accumulating behind it, crashed through all together.  There wasn't a prolonged period, or even a single moment that I remember, of wondering how I could have been so stupid.  I already knew how.

And I also knew, all at once, in the same moment of realization, that to say, I almost destroyed the world!, would have been too prideful.

It would have been too confirming of ego, too confirming of my own importance in the scheme of things, at a time when - I understood in the same moment of realization - my ego ought to be taking a major punch to the stomach.  I had been so much less than I needed to be; I had to take that punch in the stomach, not avert it.

Give it to Me Straight! I Swear I Won't be Mad!

I have an American friend (same guy as in this earlier post) who lived for a number of years in Mexico.  He married a Mexican woman, and while he always spoke to his kids in English, their real first language was Spanish.  He recently moved back to the U.S., and he enrolled his oldest daughter in kindergarten.  The school gave her some kind of language evaluation, and they concluded that she was slightly behind in English, and said they would like to give her some kind of limited special instruction if her parents wanted it.  My friend and his wife were inclined to go along with what the teachers thought, but they wanted to know the answers to a few common-sense questions: how behind was the kid really, was what they would do for her during the special instruction time really worth giving up whatever she would miss in the regular class, and so on.  The problem was, they were having a hard time getting any straight answers out of the teachers, and they were pretty sure they knew why: these very nice, well-meaning teachers were so worried about offending them that they couched every answer in a million caveats and weasel words.  My friend said he said he was dying to say something like: "I hereby unconditionally vow not to sue you, hate you, or speak or think ill of you in any way.  Now will you please just tell me what's going on with my kid?!?"

Don't get me wrong: that hyper-sensitivity comes mostly from a good place, and I certainly don't want to go back 50 years when a kid like that would just be thrown in the deep end of the pool.  But come on!

My Naturalistic Awakening

Followup to:  Fighting a Rearguard Action Against the Truth

In yesterday's episode, Eliezer₂₀₀₁ is fighting a rearguard action against the truth.  Only gradually shifting his beliefs, admitting an increasing probability in a different scenario, but never saying outright, "I was wrong before."  He repairs his strategies as they are challenged, finding new justifications for just the same plan he pursued before.

(Of which it is therefore said:  "Beware lest you fight a rearguard retreat against the evidence, grudgingly conceding each foot of ground only when forced, feeling cheated.  Surrender to the truth as quickly as you can.  Do this the instant you realize what you are resisting; the instant you can see from which quarter the winds of evidence are blowing against you.")

Memory fades, and I can hardly bear to look back upon those times - no, seriously, I can't stand reading my old writing.  I've already been corrected once in my recollections, by those who were present.  And so, though I remember the important events, I'm not really sure what order they happened in, let alone what year.

But if I had to pick a moment when my folly broke, I would pick the moment when I first comprehended, in full generality, the notion of an optimization process.  That was the point at which I first looked back and said, "I've been a fool."

Fighting a Rearguard Action Against the Truth

Followup to:  That Tiny Note of Discord, The Importance of Saying "Oops"

When we last left Eliezer₂₀₀₀, he was just beginning to investigate the question of how to inscribe a morality into an AI.  His reasons for doing this don't matter at all, except insofar as they happen to historically demonstrate the importance of perfectionism.  If you practice something, you may get better at it; if you investigate something, you may find out about it; the only thing that matters is that Eliezer₂₀₀₀ is, in fact, focusing his full-time energies on thinking technically about AI morality; rather than, as previously, finding an justification for not spending his time this way.  In the end, this is all that turns out to matter.

But as our story begins - as the sky lightens to gray and the tip of the sun peeks over the horizon - Eliezer₂₀₀₁ hasn't yet admitted that Eliezer₁₉₉₇ was mistaken in any important sense.  He's just making Eliezer₁₉₉₇'s strategy even better by including a contingency plan for "the unlikely event that life turns out to be meaningless"...

...which means that Eliezer₂₀₀₁ now has a line of retreat away from his mistake.

I don't just mean that Eliezer₂₀₀₁ can say "Friendly AI is a contingency plan", rather than screaming "OOPS!"

I mean that Eliezer₂₀₀₁ now actually has a contingency plan.  If Eliezer₂₀₀₁ starts to doubt his 1997 metaethics, the Singularity has a fallback strategy, namely Friendly AI.  Eliezer₂₀₀₁ can question his metaethics without it signaling the end of the world.

And his gradient has been smoothed; he can admit a 10% chance of having previously been wrong, then a 20% chance.  He doesn't have to cough out his whole mistake in one huge lump.

If you think this sounds like Eliezer₂₀₀₁ is too slow, I quite agree.

That Tiny Note of Discord

Followup to:  The Sheer Folly of Callow Youth

When we last left Eliezer₁₉₉₇, he believed that any superintelligence would automatically do what was "right", and indeed would understand that better than we could; even though, he modestly confessed, he did not understand the ultimate nature of morality.  Or rather, after some debate had passed, Eliezer₁₉₉₇ had evolved an elaborate argument, which he fondly claimed to be "formal", that we could always condition upon the belief that life has meaning; and so cases where superintelligences did not feel compelled to do anything in particular, would fall out of consideration.  (The flaw being the unconsidered and unjustified equation of "universally compelling argument" with "right".)

So far, the young Eliezer is well on the way toward joining the "smart people who are stupid because they're skilled at defending beliefs they arrived at for unskilled reasons".  All his dedication to "rationality" has not saved him from this mistake, and you might be tempted to conclude that it is useless to strive for rationality.

But while many people dig holes for themselves, not everyone succeeds in clawing their way back out.

And from this I learn my lesson:  That it all began -

- with a small, small question; a single discordant note; one tiny lonely thought...

The Sheer Folly of Callow Youth

Followup to:  My Childhood Death Spiral, My Best and Worst Mistake, A Prodigy of Refutation

"There speaks the sheer folly of callow youth; the rashness of an ignorance so abysmal as to be possible only to one of your ephemeral race..."
        -- Gharlane of Eddore

Once upon a time, years ago, I propounded a mysterious answer to a mysterious question - as I've hinted on several occasions.  The mysterious question to which I propounded a mysterious answer was not, however, consciousness - or rather, not only consciousness.  No, the more embarrassing error was that I took a mysterious view of morality.

I held off on discussing that until now, after the series on metaethics, because I wanted it to be clear that Eliezer₁₉₉₇ had gotten it wrong.

When we last left off, Eliezer₁₉₉₇, not satisfied with arguing in an intuitive sense that superintelligence would be moral, was setting out to argue inescapably that creating superintelligence was the right thing to do.

Well (said Eliezer₁₉₉₇) let's begin by asking the question:  Does life have, in fact, any meaning?