The Lens That Sees Its Flaws

Continuation of:  What is Evidence?

Light leaves the Sun and strikes your shoelaces and bounces off; some photons enter the pupils of your eyes and strike your retina; the energy of the photons triggers neural impulses; the neural impulses are transmitted to the visual-processing areas of the brain; and there the optical information is processed and reconstructed into a 3D model that is recognized as an untied shoelace; and so you believe that your shoelaces are untied.

Here is the secret of deliberate rationality - this whole entanglement process is not magic, and you can understand it.  You can understand how you see your shoelaces.  You can think about which sort of thinking processes will create beliefs which mirror reality, and which thinking processes will not.

Mice can see, but they can't understand seeing.  You can understand seeing, and because of that, you can do things which mice cannot do.  Take a moment to marvel at this, for it is indeed marvelous.

Mice see, but they don't know they have visual cortexes, so they can't correct for optical illusions.  A mouse lives in a mental world that includes cats, holes, cheese and mousetraps - but not mouse brains.  Their camera does not take pictures of its own lens.  But we, as humans, can look at a seemingly bizarre image, and realize that part of what we're seeing is the lens itself.  You don't always have to believe your own eyes, but you have to realize that you have eyes - you must have distinct mental buckets for the map and the territory, for the senses and reality.  Lest you think this a trivial ability, remember how rare it is in the animal kingdom.

The whole idea of Science is, simply, reflective reasoning about a more reliable process for making the contents of your mind mirror the contents of the world.  It is the sort of thing mice would never invent.  Pondering this business of "performing replicable experiments to falsify theories", we can see why it works.  Science is not a separate magisterium, far away from real life and the understanding of ordinary mortals.  Science is not something that only applies to the inside of laboratories.  Science, itself, is an understandable process-in-the-world that correlates brains with reality.

Science makes sense, when you think about it.  But mice can't think about thinking, which is why they don't have Science.  One should not overlook the wonder of this - or the potential power it bestows on us as individuals, not just scientific societies.

Admittedly, understanding the engine of thought may be a little more complicated than understanding a steam engine - but it is not a fundamentally different task.

Once upon a time, I went to EFNet's #philosophy to ask "Do you believe a nuclear war will occur in the next 20 years?  If no, why not?"  One person who answered the question said he didn't expect a nuclear war for 100 years, because "All of the players involved in decisions regarding nuclear war are not interested right now."  "But why extend that out for 100 years?", I asked. "Pure hope," was his reply.

Reflecting on this whole thought process, we can see why the thought of nuclear war makes the person unhappy, and we can see how his brain therefore rejects the belief.  But, if you imagine a billion worlds - Everett branches, or Tegmark duplicates - this thought process will not systematically correlate optimists to branches in which no nuclear war occurs.  (Some clever fellow is bound to say, "Ah, but since I have hope, I'll work a little harder at my job, pump up the global economy, and thus help to prevent countries from sliding into the angry and hopeless state where nuclear war is a possibility.  So the two events are related after all."  At this point, we have to drag in Bayes's Theorem and measure the charge of entanglement quantitatively.  Your optimistic nature cannot have that large an effect on the world; it cannot, of itself, decrease the probability of nuclear war by 20%, or however much your optimistic nature shifted your beliefs.  Shifting your beliefs by a large amount, due to an event that only carries a very tiny charge of entanglement, will still mess up your mapping.)

To ask which beliefs make you happy, is to turn inward, not outward - it tells you something about yourself, but it is not evidence entangled with the environment.  I have nothing anything against happiness, but it should follow from your picture of the world, rather than tampering with the mental paintbrushes.

If you can see this - if you can see that hope is shifting your first-order thoughts by too large a degree - if you can understand your mind as a mapping-engine with flaws in it - then you can apply a reflective correction.  The brain is a flawed lens through which to see reality.  This is true of both mouse brains and human brains.  But a human brain is a flawed lens that can understand its own flaws - its systematic errors, its biases - and apply second-order corrections to them.  This, in practice, makes the flawed lens far more powerful.  Not perfect, but far more powerful.

Comments

[Hopefully Anonymous]

Eliezer, nice to read your opinions on a whole range of things in this post, but I think it would be more helpful to us for you to not state your opinions as fact (and to not state overcertainty about the existence and mechanics of various phenomena).

[Stuart Armstrong]

Light leaves the Sun and strikes your shoelaces and bounces off; some photons enter the pupils of your eyes and strike your retina; the energy of the photons triggers neural impulses; the neural impulses are transmitted to the visual-processing areas of the brain; and there the optical information is processed and reconstructed into a 3D model that is recognized as an untied shoelace; and so you believe that your shoelaces are untied.

Here is the secret of deliberate rationality - this whole entanglement process is not magic, and you can understand it.

But if we were minds in a vat, or cogs in the Matrix, we would still be able to reason rationally and make intelligent predictions about the world we see. And test them, and improve our predictions and discard the ones that are wrong. We can be rational about the real world, even if the real world is an illusion.

So I don't see how we can found rationality on our understanding of the world (a world we only understand through reason). In this argument, where is the egg that was not born of a chicken?

[Eliezer Yudkowsky]

HA, your objection is too vague for me to apply. Specify.

But if we were minds in a vat, or cogs in the Matrix, we would still be able to reason rationally and make intelligent predictions about the world we see. And test them, and improve our predictions and discard the ones that are wrong. We can be rational about the real world, even if the real world is an illusion.

I do not understand your bizarre concept, illusion. Whatever is, is real. Sometimes the underlying levels of organization are different from what you expected.

So I don't see how we can found rationality on our understanding of the world (a world we only understand through reason). In this argument, where is the egg that was not born of a chicken?

That's why I distinguished deliberate rationality. Seeing your shoelaces is also rational, for it produces beliefs that are themselves evidence; but it is not a process that requires deliberate control. The lens sees, even in mice; but only in humans does the lens see itself and see its flaws.

[Rolf Nelson]

Re: wishful thinking, I've personally seen this before, where people *explicitly* reject reason on an important topic; I knew a rabbi in Minnesota who insisted the Israeli-Palestinian peace process will succeed, simply because "it *must* succeed." Usually people only explicitly reject reason on "one thought too many" topics like "I would never even think about betraying my friends", but the wishful-thinking topics such as your nuclear-war example don't seem to fit into this mold.

Anyone know what the research says on this? I know people faced with death will shift their *values*, but to what degree and in what directions do they shift their estimated probability of deaths and disasters when the disaster involves them or people they care about? And is this just part of a more general wishful-thinking bias? (Not that I know what the research says about wishful thinking, either.)

Conjecture: a New Yorker is more likely to see D.C. as the likely first target for a terrorist nuclear bomb, compared with a D.C. resident.

[Hopefully Anonymous]

Eliezer,

Here's a good example in your reply to Stuart: "but only in humans does the lens see itself and see its flaws". Here I think, as in my previous critical post, that you're "stat[ing] overcertainty about the existence and mechanics of various phenomena".

One might say that writing these statements in a more provisional and tentative fashion, such as "As far as we can tell, some humans are the only things capable of analyzing flaws in their ability to to observe the universe, and pointing out this exceptionalist element about some humans is of use because of X" makes communication too cumbersome, and there's no need to to say because such nuances are implied.

But I disagree. I think the overcertain style of writing you and some other commenters fall into is less helpful for discussing this stuff than a greater level of nuance, and framing ideas and knowledge more provisionally.

In short, I'm requesting greater transparency about our bounded rationality in your posts.

[Eliezer Yudkowsky]

HA, it is indeed too cumbersome. See also Orwell's "Politics and the English Language."

Ad hominem tu quoque: You didn't rewrite your own comment in the cumbersome style you wanted me to use. In fact, your initial comment was so extremely minimal that I couldn't apply it, and not qualified at all.

[ungoverned]

The "lens" sees perhaps only parts of itself, and then perhaps only some of its flaws.

[Cure of Ars]

For hope to be useless, it requires the premise that God does not exist. If God exists, then the rational thing is to hope and not in just the improbable but the impossible.

As a Catholic, I am willing to abstain from food and sex at times. I even like to think that I would give my life for my faith. But you atheists are fanatical. Sacrificing hope is too hardcore. First you sacrifice faith, then hope, what’s next love?

[g]

If it is true that "if God exists, then the rational thing is to hope and not in just the improbable but the impossible", then that fact is itself strong evidence against the existence of God.

But who said anything about sacrificing hope? Eliezer argues against wishful thinking, which is not at all the same thing as hope. Oh, and the idea that "faith, hope and love" are the same kind of thing -- so much the same kind of thing that abandoning two of them would be likely to lead to abandoning the third -- seems to me to have no support at all outside the First Letter to the Corinthians; why should Eliezer fear that abandoning faith and (what you rather bizarrely call) hope should lead to abandoning love?

[Cure of Ars]

Eliezer argues against wishful thinking, which is not at all the same thing as hope.

You characterize my understanding of hope as “wishful thinking”. I would characterize your understanding of hope as mere placebo. A placebo that does not amount to a hill of beans. If there is no God than you are correct that my understanding of hope is wishful thinking. If there is a God then my understanding of hope is rational. It is also true that if there is a God that your understanding of hope is woefully insufficient.

Oh, and the idea that "faith, hope and love" are the same kind of thing -- so much the same kind of thing that abandoning two of them would be likely to lead to abandoning the third -- seems to me to have no support at all outside the First Letter to the Corinthians; why should Eliezer fear that abandoning faith and (what you rather bizarrely call) hope should lead to abandoning love?

Faith, hope and love are the Christian theological virtues. I would argue that they are at the core of what it is to live a fully human life. It looks like this website has rejected the theistic understanding of faith and hope. I don’t see what is stopping the rejection of love due to it being a strong biasing factor. I don’t know how you can love something without it making you biased towards it. To really be unbiased we should not love humanity and in so doing the logical conclusion is that man is insignificant. What we are does not matter in the scope of time and space. You may not like my conclusion but I don’t see how it does not follow from the atheistic premises that this website holds.

[Tom McCabe]

"I would argue that they are at the core of what it is to live a fully human life."

A fully human life, in the natural sense of the term, has an average span of sixteen years. That's the environment we were designed to live in- nasty, brutal, and full of misery. By the standards of a typical human tribe, the Holocaust would have been notable for killing such a remarkably small percentage of the population. Why on Earth would we want to follow that example?

"It looks like this website has rejected the theistic understanding of faith and hope."

Yes, for a very good reason- it does not work. If you stand in front of a truck, and you have faith that the truck will not run you over, and you hope that the truck will not run you over, your bones and vital organs will be sliced and diced and chopped and fried. The key factor in survival is not lack of hope, or lack of faith, but lack of doing stupid things such as standing in front of trucks.

"I don’t know how you can love something without it making you biased towards it."

This is not what we mean by "biased". By "bias", we mean bugs in the human brain which lead us to give wrong answers to simple questions of fact, such as "What is the probability of X?". See http://www.acceleratingfuture.com/tom/?p=30.

[James Blair]

Faith, hope and love are the Christian theological virtues.

What about other religions? Islam and Judaism come to mind, but there are also non-abrahamic religions that advocate faith, hope and love. Why is are you exclusively a Christian and not a Muslim, a Jew, a Buddhist or a Pagan? Why are you a Catholic instead of a Protestant? If you were born in China in the early 20th century, would you be a Catholic? If so, why? If not, why are you a Catholic here and now?

[Cure of Ars]

A fully human life, in the natural sense of the term, has an average span of sixteen years. That's the environment we were designed to live in- nasty, brutal, and full of misery. By the standards of a typical human tribe, the Holocaust would have been notable for killing such a remarkably small percentage of the population. Why on Earth would we want to follow that example?

In a lot of ways we don’t have a shared vocabulary. When I said fully human life I was not using this in the natural sense. Our understanding of humanity is different. I see man as made in the image of God. You see man as just another animal that is a product of evolutionary mechanisms. I guess the closest secular term that I can use to convey what I am saying is Maslow's self actualization but transcendent.

Yes, for a very good reason- it does not work. If you stand in front of a truck, and you have faith that the truck will not run you over, and you hope that the truck will not run you over, your bones and vital organs will be sliced and diced and chopped and fried. The key factor in survival is not lack of hope, or lack of faith, but lack of doing stupid things such as standing in front of trucks.

Sure God is not going to change natural law just because we are putting him to the test. Twelve poor followers of Christ were able to convert the Roman empire. I have a hard time believing the virtue of hope was not involved. I could go into the lives of the saints for other examples but I wont.

This is not what we mean by "biased". By "bias", we mean bugs in the human brain which lead us to give wrong answers to simple questions of fact, such as "What is the probability of X?". See http://www.acceleratingfuture.com/tom/?p=30.

You call the getting to the probability of nuclear war a simple question? Why doesn’t love lead to bugs in the human brain that leads us to wrong answers?

[Eliezer Yudkowsky]

Cure, you're making too many comments. A good rule of thumb is that you should never have made more than 3 and preferably 2 of the 10 most recent comments. You've made it clear what you believe; everyone knows you're a Catholic now; you do not need to repeat it further.

[Tom McCabe]

"I see man as made in the image of God."

This does make some sense. If man is made in the image of God, and we know God is a mass murderer, then we can predict that some men will also be mass murderers. And lo, we have plenty of examples- Hitler, Stalin, Mao, Pol Pot, etc.

"Sure God is not going to change natural law just because we are putting him to the test."

If God does exist, as soon as we finish saving the world and whatnot, he should be immediately arrested and put on trial for crimes against humanity, due to his failure to intervene in the Holocaust, the smallpox epidemics, WWI, etc.

"Twelve poor followers of Christ were able to convert the Roman empire."

Aye. And Karl Marx must have had divine powers too- how else could a single person, with no political authority, cause a succession of revolutions in some of the largest countries on Earth?

"I could go into the lives of the saints for other examples but I wont."

How do you know that large parts of their lives weren't simply made up?

"You call the getting to the probability of nuclear war a simple question?"

Read the literature on heuristics and biases- researchers deliberately use simple questions with factual answers, so that the data unambiguously show the flaws in human reasoning.

[g]

CoA, if you "would argue that [faith, hope and love] are at the core of what it is to live a fully human life" then why don't you, rather than just asserting it? (Or, if the argument you'd make is much too long and convoluted, point us to somewhere where it's made in a non-question-begging way.)

"This website" doesn't reject anything. It can't. It's only a website. A lot of the posters and commenters here disagree with "the theistic understanding of faith and hope", but people who think otherwise aren't forbidden to contribute or anything.

Tom, CoA isn't saying "the apostles converted everyone to Christianity, so it must have been a miracle" (though he may well believe it); he's saying "Christianity took over much of the world from tiny beginnings; it seems likely that the people involved were more optimistic than the evidence would have seemed to warrant". He's probably right about that (see "Small Business Overconfidence"). The same is surely true of at least some of the people involved in the rise of communism. Optimism beyond the evidence probably *is* an advantage, if your goal is to have a belief that isn't well supported by the evidence become hugely popular and influential. Demagogues and revolutionaries and medical quacks all tend to be optimistic beyond the evidence.