Overcoming Bias

Are you dreaming?

Often when I’m dreaming I “feel” that I’m awake.  When I’m awake, however, I always  “feel” that I’m awake and have no conscious doubt (except in the philosophical sense) that I’m not dreaming.

But logically when I “feel” awake I should believe there is a non-trivial chance that I’m dreaming.  This has implications for how I should behave.

For example, imagine I’m considering eating spinach or chocolate.  I like the taste of chocolate more than spinach, but recognize that spinach is healthier for me.  Let’s say that if the probability of my being awake were greater than 99% then to maximize the expected overall quality of my life I should eat the spinach otherwise I should pick the chocolate. 

Rationally, I should probably figure that the chance of my being awake is less than 99% so I should go with the chocolate.  Yet like most other humans I don’t take into account that I might be dreaming when I “feel” awake.

Over the long run you would likely reduce your inclusive genetic fitness if when you  “feel” awake you act as if there is a less than 100% chance of your actually being awake.  For this reason I suspect we are “genetically programmed” to never doubt that we are awake when we “feel” awake even though it would be rational to hold such a doubt.

Ask OB: Leaving the Fold

Followup to:  Crisis of Faith

I thought this comment from "Jo" deserved a bump to the front page:

"So here I am having been raised in the Christian faith and trying not to freak out over the past few weeks because I've finally begun to wonder whether I believe things just because I was raised with them. Our family is surrounded by genuinely wonderful people who have poured their talents into us since we were teenagers, and our social structure and business rests on the tenets of what we believe. I've been trying to work out how I can 'clear the decks' and then rebuild with whatever is worth keeping, yet it's so foundational that it will affect my marriage (to a pretty special man) and my daughters who, of course, have also been raised to walk the Christian path.

Is there anyone who's been in this position - really, really invested in a faith and then walked away?"

Handling this kind of situation has to count as part of the art.  But I haven't gone through anything like this.  Can anyone with experience advise Jo on what to expect, what to do, and what not to do?

Back Up and Ask Whether, Not Why

Followup to:  The Bottom Line

A recent conversation reminded me of this simple, important, and difficult method:

When someone asks you "Why are you doing X?",
And you don't remember an answer previously in mind,
Do not ask yourself "Why am I doing X?".

For example, if someone asks you
"Why are you using a QWERTY keyboard?" or "Why haven't you invested in stocks?"
and you don't remember already considering this exact question and deciding it,
do not ask yourself "Why am I using a QWERTY keyboard?" or "Why aren't I invested in stocks?"

Instead, try to blank your mind - maybe not a full-fledged crisis of faith, but at least try to prevent your mind from knowing the answer immediately - and ask yourself:

"Should I do X, or not?"

Should I use a QWERTY keyboard, or not?  Should I invest in stocks, or not?

When you finish considering this question, print out a traceback of the arguments that you yourself considered in order to arrive at your decision, whether that decision is to X, or not X.  Those are your only real reasons, nor is it possible to arrive at a real reason in any other way.

And this is also writing advice: because I have sometimes been approached by people who say "How do I convince people to wear green shoes?  I don't know how to argue it," and I reply, "Ask yourself honestly whether you should wear green shoes; then make a list of which thoughts actually move you to decide one way or another; then figure out how to explain or argue them, recursing as necessary."

Trust But Don't Verify

Aimone and Houser find we are willing to pay to avoid knowing that we have been betrayed: 

Here we report data from one-shot two-person binary investment games in which investors can choose not to know the decision of their particular trustee, and instead receive payment according to a random draw from a separate pool of decisions identical to the pool of trustees' decisions. Note that the probability of receiving the "cooperative" outcome is identical in the two cases, and participants understand this is the case. ... Our main finding is that investors systematically prefer to remain ignorant of their specific trustee's decision. Moreover, when avoiding this information is not possible investors are substantially less likely to make trusting decisions. These results are convergent evidence that outcome-based models cannot fully explain economic decision making in strategic environments.

Added 25 Oct: Can this help explain why we rely so little on incentive contracts for docs, real estate agents, etc.? 

Dark Side Epistemology

Followup to:  Entangled Truths, Contagious Lies

If you once tell a lie, the truth is ever after your enemy.

I have previously spoken of the notion that, the truth being entangled, lies are contagious.  If you pick up a pebble from the driveway, and tell a geologist that you found it on a beach - well, do you know what a geologist knows about rocks?  I don't.  But I can suspect that a water-worn pebble wouldn't look like a droplet of frozen lava from a volcanic eruption.  Do you know where the pebble in your driveway really came from?  Things bear the marks of their places in a lawful universe; in that web, a lie is out of place.

What sounds like an arbitrary truth to one mind - one that could easily be replaced by a plausible lie - might be nailed down by a dozen linkages to the eyes of greater knowledge.  To a creationist, the idea that life was shaped by "intelligent design" instead of "natural selection" might sound like a sports team to cheer for.  To a biologist, plausibly arguing that an organism was intelligently designed would require lying about almost every facet of the organism.  To plausibly argue that "humans" were intelligently designed, you'd have to lie about the design of the human retina, the architecture of the human brain, the proteins bound together by weak van der Waals forces instead of strong covalent bonds...

Or you could just lie about evolutionary theory, which is the path taken by most creationists.  Instead of lying about the connected nodes in the network, they lie about the general laws governing the links.

And then to cover that up, they lie about the rules of science - like what it means to call something a "theory", or what it means for a scientist to say that they are not absolutely certain.

Ends Don't Justify Means (Among Humans)

Followup to:  Why Does Power Corrupt?

"If the ends don't justify the means, what does?"
        -- variously attributed

"I think of myself as running on hostile hardware."
        -- Justin Corwin

Yesterday I talked about how humans may have evolved a structure of political revolution, beginning by believing themselves morally superior to the corrupt current power structure, but ending by being corrupted by power themselves - not by any plan in their own minds, but by the echo of ancestors who did the same and thereby reproduced.

This fits the template:

In some cases, human beings have evolved in such fashion as to think that they are doing X for prosocial reason Y, but when human beings actually do X, other adaptations execute to promote self-benefiting consequence Z.

From this proposition, I now move on to my main point, a question considerably outside the realm of classical Bayesian decision theory:

"What if I'm running on corrupted hardware?"

Why Does Power Corrupt?

Followup to:  Evolutionary Psychology

"Power tends to corrupt, and absolute power corrupts absolutely.  Great men are almost always bad men."
        -- Lord Acton

Call it a just-so story if you must, but as soon as I was introduced to the notion of evolutionary psychology (~1995), it seemed obvious to me why human beings are corrupted by power.  I didn't then know that hunter-gatherer bands tend to be more egalitarian than agricultural tribes - much less likely to have a central tribal-chief boss-figure - and so I thought of it this way:

Humans (particularly human males) have evolved to exploit power and status when they obtain it, for the obvious reason:  If you use your power to take many wives and favor your children with a larger share of the meat, then you will leave more offspring, ceteris paribus.  But you're not going to have much luck becoming tribal chief if you just go around saying, "Put me in charge so that I can take more wives and favor my children."  You could lie about your reasons, but human beings are not perfect deceivers.

So one strategy that an evolution could follow, would be to create a vehicle that reliably tended to start believing that the old power-structure was corrupt, and that the good of the whole tribe required their overthrow...

The Ritual

Followup to:  The Failures of Eld Science, Crisis of Faith

The room in which Jeffreyssai received his non-beisutsukai visitors was quietly formal, impeccably appointed in only the most conservative tastes.  Sunlight and outside air streamed through a grillwork of polished silver, a few sharp edges making it clear that this wall was not to be opened.  The floor and walls were glass, thick enough to distort, to a depth sufficient that it didn't matter what might be underneath.  Upon the surfaces of the glass were subtly scratched patterns of no particular meaning, scribed as if by the hand of an artistically inclined child (and this was in fact the case).

Elsewhere in Jeffreyssai's home there were rooms of other style; but this, he had found, was what most outsiders expected of a Bayesian Master, and he chose not to enlighten them otherwise.  That quiet amusement was one of life's little joys, after all.

The guest sat across from him, knees on the pillow and heels behind.  She was here solely upon the business of her Conspiracy, and her attire showed it:  A form-fitting jumpsuit of pink leather with even her hands gloved - all the way to the hood covering her head and hair, though her face lay plain and unconcealed beneath.

And so Jeffreyssai had chosen to receive her in this room.

Jeffreyssai let out a long breath, exhaling.  "Are you sure?"

"Oh," she said, "and do I have to be absolutely certain before my advice can shift your opinions?  Does it not suffice that I am a domain expert, and you are not?"

Crisis of Faith

Followup to:  Make an Extraordinary Effort, The Meditation on Curiosity, Avoiding Your Belief's Real Weak Points

"It ain't a true crisis of faith unless things could just as easily go either way."
        - Thor Shenkel

Many in this world retain beliefs whose flaws a ten-year-old could point out, if that ten-year-old were hearing the beliefs for the first time.  These are not subtle errors we are talking about.  They would be child's play for an unattached mind to relinquish, if the skepticism of a ten-year-old were applied without evasion. As Premise Checker put it, "Had the idea of god not come along until the scientific age, only an exceptionally weird person would invent such an idea and pretend that it explained anything."

And yet skillful scientific specialists, even the major innovators of a field, even in this very day and age, do not apply that skepticism successfully.  Nobel laureate Robert Aumann, of Aumann's Agreement Theorem, is an Orthodox Jew:  I feel reasonably confident in venturing that Aumann must, at one point or another, have questioned his faith.  And yet he did not doubt successfully.  We change our minds less often than we think.

This should scare you down to the marrow of your bones.  It means you can be a world-class scientist and conversant with Bayesian mathematics and still fail to reject a belief whose absurdity a fresh-eyed ten-year-old could see.  It shows the invincible defensive position which a belief can create for itself, if it has long festered in your mind.

What does it take to defeat an error which has built itself a fortress?

But by the time you know it is an error, it is already defeated.  The dilemma is not "How can I reject long-held false belief X?" but "How do I know if long-held belief X is false?"  Self-honesty is at its most fragile when we're not sure which path is the righteous one.  And so the question becomes:

How can we create in ourselves a true crisis of faith, that could just as easily go either way?

Shut up and do the impossible!

Followup to:  Make An Extraordinary Effort, On Doing the Impossible, Beyond the Reach of God

The virtue of tsuyoku naritai, "I want to become stronger", is to always keep improving - to do better than your previous failures, not just humbly confess them.

Yet there is a level higher than tsuyoku naritai.  This is the virtue of isshokenmei, "make a desperate effort".  All-out, as if your own life were at stake.  "In important matters, a 'strong' effort usually only results in mediocre results."

And there is a level higher than isshokenmei.  This is the virtue I called "make an extraordinary effort".  To try in ways other than what you have been trained to do, even if it means doing something different from what others are doing, and leaving your comfort zone.  Even taking on the very real risk that attends going outside the System.

But what if even an extraordinary effort will not be enough, because the problem is impossible?

I have already written somewhat on this subject, in On Doing the Impossible.  My younger self used to whine about this a lot:  "You can't develop a precise theory of intelligence the way that there are precise theories of physics.  It's impossible!  You can't prove an AI correct.  It's impossible!  No human being can comprehend the nature of morality - it's impossible!  No human being can comprehend the mystery of subjective experience!  It's impossible!"

And I know exactly what message I wish I could send back in time to my younger self:

Shut up and do the impossible!