Overcoming Bias

That Tiny Note of Discord

Followup to:  The Sheer Folly of Callow Youth

When we last left Eliezer₁₉₉₇, he believed that any superintelligence would automatically do what was "right", and indeed would understand that better than we could; even though, he modestly confessed, he did not understand the ultimate nature of morality.  Or rather, after some debate had passed, Eliezer₁₉₉₇ had evolved an elaborate argument, which he fondly claimed to be "formal", that we could always condition upon the belief that life has meaning; and so cases where superintelligences did not feel compelled to do anything in particular, would fall out of consideration.  (The flaw being the unconsidered and unjustified equation of "universally compelling argument" with "right".)

So far, the young Eliezer is well on the way toward joining the "smart people who are stupid because they're skilled at defending beliefs they arrived at for unskilled reasons".  All his dedication to "rationality" has not saved him from this mistake, and you might be tempted to conclude that it is useless to strive for rationality.

But while many people dig holes for themselves, not everyone succeeds in clawing their way back out.

And from this I learn my lesson:  That it all began -

- with a small, small question; a single discordant note; one tiny lonely thought...

Say It Loud

Reply to:  Overconfidence is Stylish

I respectfully defend my lord Will Strunk:

"If you don't know how to pronounce a word, say it loud! If you don't know how to pronounce a word, say it loud!"  This comical piece of advice struck me as sound at the time, and I still respect it. Why compound ignorance with inaudibility?  Why run and hide?

How does being vague, tame, colorless, irresolute, help someone to understand your current state of uncertainty?  Any more than mumbling helps them understand a word you aren't sure how to pronounce?

Goofus says:  "The sky, if such a thing exists at all, might or might not have a property of color, but, if it does have color, then I feel inclined to state that it might be green."

Gallant says:   "70% probability the sky is green."

Which of them sounds more confident, more definite?

But which of them has managed to quickly communicate their state of uncertainty?

(And which of them is more likely to actually, in real life, spend any time planning and preparing for the eventuality that the sky is blue?)

A Genius for Destruction

This is a question from a workshop after the Global Catastrophic Risks conference.  The rule of the workshop was that people could be quoted, but not attributed, so I won't say who observed:

"The problem is that it's often our smartest people leading us into the disasters.  Look at Long-Term Capital Management."

To which someone else replied:

"Maybe smart people are just able to work themselves up into positions of power, so that if damage gets caused, the responsibility will often lie with someone smart."

Experience Increases Overconfidence

The latest Journal of Prediction Markets has a lit review on overconfidence, with a to-me surprising result: financial overconfidence increases with age, experience, and success.  Here are three investment experiments:

Kirchler and Maciejovsky (2002) .... demonstrated that subjects were overconfident in late trading periods ... [but] underconfident during other trading periods. ... Dittrich et al (2005) ... found that age was positively correlated to overconfidence for complex tasks.  ... Glaser et al (2005) ... [found that financial market] professionals' degrees of overconfidence was higher than that of the student subjects in most tasks, and it appeared that was because the "professionals are biased in job related tasks, such as forecasting real world financial time series."

Also:

A survey of German stock market forecasters conducted by Deaves et al (2005) ... demonstrated that the market forecasters were overconfident in their predictions and that greater market experience and success, measured by correct predictions, increased their overconfidence.  ... Markets are likely to become more overconfident when market returns are high. 

This is disturbing.  If overconfidence is positively correlated with ability, then observers can rationally take overconfidence as a signal of ability, and people can want to appear more overconfident to appear more able.  But to make this story work, somehow it should on average be easier to get away with being more overconfident when one is more experienced and successful.  How can this be?

This all seems to make it more reasonable than one might otherwise have thought to disagree about finance with older, more experienced, more successful folks.

Joe Epstein on Youth

More on our overconfident kids from a thoughful essay by Joseph Epstein:

So often in my literature classes students told me what they "felt" about a novel, or a particular character in a novel. I tried, ever so gently, to tell them that no one cared what they felt; the trick was to discover not one's feelings but what the author had put into the book, its moral weight and its resultant power. In essay courses, many of these same students turned in papers upon which I wished to--but did not--write: "D-, Too much love in the home." I knew where they came by their sense of their own deep significance and that this sense was utterly false to any conceivable reality. Despite what their parents had been telling them from the very outset of their lives, they were not significant. Significance has to be earned, and it is earned only through achievement. Besides, one of the first things that people who really are significant seem to know is that, in the grander scheme, they are themselves really quite insignificant.

Against Devil's Advocacy

From an article by Michael Ruse:

Richard Dawkins once called me a "creep." He did so very publicly but meant no personal offense, and I took none: We were, and still are, friends. The cause of his ire—his anguish, even—was that, in the course of a public discussion, I was defending a position I did not truly hold. We philosophers are always doing this; it's a version of the reductio ad absurdum argument. We do so partly to stimulate debate (especially in the classroom), partly to see how far a position can be pushed before it collapses (and why the collapse), and partly (let us be frank) out of sheer bloody-mindedness, because we like to rile the opposition.

Dawkins, however, has the moral purity—some would say the moral rigidity—of the evangelical Christian or the committed feminist. Not even for the sake of argument can he endorse something that he thinks false. To do so is not just mistaken, he feels; in some deep sense, it is wrong. Life is serious, and there are evils to be fought. There must be no compromise or equivocation, even for pedagogical reasons. As the Quakers say, "Let your yea be yea, and your nay, nay."

Michael Ruse doesn't get it.

Overconfidence & Paternalism

Paul Graham tries to explain paternalism: 

Parents know they've concealed the facts about sex, and many at some point sit their kids down and explain more. But few tell their kids about the differences between the real world and the cocoon they grew up in. Combine this with the confidence parents try to instill in their kids, and every year you get a new crop of 18 year olds who think they know how to run the world.

Don't all 18 year olds think they know how to run the world? Actually this seems to be a recent innovation, no more than about 100 years old. In preindustrial times teenage kids were junior members of the adult world and comparatively well aware of their shortcomings. They could see they weren't as strong or skillful as the village smith. In past times people lied to kids about some things more than we do now, but the lies implicit in an artificial, protected environment are a recent invention. Like a lot of new inventions, the rich got this first. Children of kings and great magnates were the first to grow up out of touch with the world. Suburbia means half the population can live like kings in that respect.  ...

Science Doesn't Trust Your Rationality

Followup to:  The Dilemma: Science or Bayes?

Scott Aaronson suggests that Many-Worlds and libertarianism are similar in that they are both cases of bullet-swallowing, rather than bullet-dodging:

Libertarianism and MWI are both are grand philosophical theories that start from premises that almost all educated people accept (quantum mechanics in the one case, Econ 101 in the other), and claim to reach conclusions that most educated people reject, or are at least puzzled by (the existence of parallel universes / the desirability of eliminating fire departments).

Now there's an analogy that would never have occurred to me.

I've previously argued that Science rejects Many-Worlds but Bayes accepts it.  (Here, "Science" is capitalized because we are talking about the idealized form of Science, not just the actual social process of science.)

It furthermore seems to me that there is a deep analogy between (small-'l') libertarianism and Science:

1. Both are based on a pragmatic distrust of reasonable-sounding arguments.
2. Both try to build systems that are more trustworthy than the people in them.
3. Both accept that people are flawed, and try to harness their flaws to power the system.

Sincerity Is Overrated

Consider choices like:

• Do I push folks at my large company to hire my son?
• Do I spend college money from my parents to pursue an acting career?
• Do cut open this patient to try my new surgical technique?

Such choices might be justified if, for example,

• My son is really good fit for the job opening.
• I have an excellent chance to succeed in acting.
• This is a very promising surgical technique.

But when am I justified in having such beliefs?  Most people think they are justified in acting on a belief if that belief is "sincere."  And by "sincere" they mean they are not conscious of just pretending to believe.  When they go to the shelf in their mind where that belief is suppose to sit, this is what they find.  And they don't remember anything illicit about how that belief got there.

But sincerity is way too low a standard!  Since humans have an enormous tendency toward self-deception, wishful thinking, and so on, we are clearly "sincerely" biased in many ways.  So to be justified in acting on a belief, you must have tried to identify and overcome relevant biases.  Furthermore, your efforts should be proportionate to the magnitude of the actions being considered, and to the magnitude of the biases that could distort your beliefs.  For important actions where biases tend to be large, you must try very hard to consider what you might have seen and felt if the world were other than you think it is. 

Knowing your argumentative limitations, OR "one [rationalist's] modus ponens is another's modus tollens."

Followup to: Who Told You Moral Questions Would be Easy?.  Response to: Circular Altruism

At the most basic level (which is all we need for present purposes), an argument is nothing but a chain of dependence between two or more propositions.  We say something about the truth value of the set of propositions {P1...Pn}, and we assert that there's something about {P1...Pn} such that if we're right about the truth values of that set, we ought to believe something about the truth value of the set {Q1...Qn}. 

If we have that understanding of what it means to make an argument, then we can see that an argument doesn't necessarily have any connection to the universe outside itself.  The utterance "1. all bleems are quathes, 2. the youiine is a bleem, 3. therefore, the youiine is a quathe" is a perfectly logically valid utterance, but it doesn't refer to anything in the world -- it doesn't require us to change any beliefs.  The meaning of any argument is conditional on our extra-argument beliefs about the world.

One important use of this principle is reflected in the oft-quoted line "one man's modus ponens in another man's modus tollens."  Modus ponens is a classical form of argument: 1. A-->B.  2.  A. 3.  .: B.  Modus tollens is this: 1.  A-->B.  2. ¬B.  3. .: ¬A.  Both are perfectly valid forms of argument!  (For those who aren't familiar with the standard notation, the horizontal line is meant to indicate negation.)  Unless you have some particular reason outside the argument to believe either A or B, you don't know whether the claim A-->B means that B is true, or that A isn't true! 

Why am I elucidating all this basic logic, which almost everyone reading this blog doubtless knows?  It's a rhetorical tactic: I'm trying to make it salient, to bring it to the top of the cognitive stack, so that my next claim is more compelling.

And that claim is as follows:

Eliezer's posts about the specks and the torture [1] [2], and the googleplex of people being tortured for a nanosecond, and so on, and so forth, tell you nothing about the truth of your intuitions.

Argument behind the fold...