Overcoming Bias

Horrible LHC Inconsistency

Followup to: When (Not) To Use Probabilities, How Many LHC Failures Is Too Many?

While trying to answer my own question on "How Many LHC Failures Is Too Many?" I realized that I'm horrendously inconsistent with respect to my stated beliefs about disaster risks from the Large Hadron Collider.

First, I thought that stating a "one-in-a-million" probability for the Large Hadron Collider destroying the world was too high, in the sense that I would much rather run the Large Hadron Collider than press a button with a known 1/1,000,000 probability of destroying the world.

But if you asked me whether I could make one million statements of authority equal to "The Large Hadron Collider will not destroy the world", and be wrong, on average, around once, then I would have to say no.

Unknown pointed out that this turns me into a money pump.  Given a portfolio of a million existential risks to which I had assigned a "less than one in a million probability", I would rather press the button on the fixed-probability device than run a random risk from this portfolio; but would rather take any particular risk in this portfolio than press the button.

Then, I considered the question of how many mysterious failures at the LHC it would take to make me question whether it might destroy the world/universe somehow, and what this revealed about my prior probability.

If the failure probability had a known 50% probability of occurring from natural causes, like a quantum coin or some such... then I suspect that if I actually saw that coin come up heads 20 times in a row, I would feel a strong impulse to bet on it coming up heads the next time around.  (And that's taking into account my uncertainty about whether the anthropic principle really works that way.)

Even having noticed this triple inconsistency, I'm not sure in which direction to resolve it!

(But I still maintain my resolve that the LHC is not worth expending political capital, financial capital, or our time to shut down; compared with using the same capital to worry about superhuman intelligence or nanotechnology.)

Optimization

"However many ways there may be of being alive, it is certain that there are vastly more ways of being dead."
        -- Richard Dawkins

In the coming days, I expect to be asked:  "Ah, but what do you mean by 'intelligence'?"  By way of untangling some of my dependency network for future posts, I here summarize some of my notions of "optimization".

Consider a car; say, a Toyota Corolla.  The Corolla is made up of some number of atoms; say, on the rough order of 10²⁹.  If you consider all possible ways to arrange 10²⁹ atoms, only an infinitesimally tiny fraction of possible configurations would qualify as a car; if you picked one random configuration per Planck interval, many ages of the universe would pass before you hit on a wheeled wagon, let alone an internal combustion engine.

Even restricting our attention to running vehicles, there is an astronomically huge design space of possible vehicles that could be composed of the same atoms as the Corolla, and most of them, from the perspective of a human user, won't work quite as well.  We could take the parts in the Corolla's air conditioner, and mix them up in thousands of possible configurations; nearly all these configurations would result in a vehicle lower in our preference ordering, still recognizable as a car but lacking a working air conditioner.

So there are many more configurations corresponding to nonvehicles, or vehicles lower in our preference ranking, than vehicles ranked greater than or equal to the Corolla.

Similarly with the problem of planning, which also involves hitting tiny targets in a huge search space.  Consider the number of possible legal chess moves versus the number of winning moves.

Which suggests one theoretical way to measure optimization - to quantify the power of a mind or mindlike process:

The Truly Iterated Prisoner's Dilemma

Followup to:  The True Prisoner's Dilemma

For everyone who thought that the rational choice in yesterday's True Prisoner's Dilemma was to defect, a follow-up dilemma:

Suppose that the dilemma was not one-shot, but was rather to be repeated exactly 100 times, where for each round, the payoff matrix looks like this:

	Humans: C	Humans:  D
Paperclipper: C	(2 million human lives saved, 2 paperclips gained)	(+3 million lives, +0 paperclips)
Paperclipper: D	(+0 lives, +3 paperclips)	(+1 million lives, +1 paperclip)

As most of you probably know, the king of the classical iterated Prisoner's Dilemma is Tit for Tat, which cooperates on the first round, and on succeeding rounds does whatever its opponent did last time.  But what most of you may not realize, is that, if you know when the iteration will stop, Tit for Tat is - according to classical game theory - irrational.

Why?  Consider the 100th round.  On the 100th round, there will be no future iterations, no chance to retaliate against the other player for defection.  Both of you know this, so the game reduces to the one-shot Prisoner's Dilemma.  Since you are both classical game theorists, you both defect.

Now consider the 99th round.  Both of you know that you will both defect in the 100th round, regardless of what either of you do in the 99th round.  So you both know that your future payoff doesn't depend on your current action, only your current payoff.  You are both classical game theorists.  So you both defect.

Now consider the 98th round...

With humanity and the Paperclipper facing 100 rounds of the iterated Prisoner's Dilemma, do you really truly think that the rational thing for both parties to do, is steadily defect against each other for the next 100 rounds?

The True Prisoner's Dilemma

It occurred to me one day that the standard visualization of the Prisoner's Dilemma is fake.

The core of the Prisoner's Dilemma is this symmetric payoff matrix:

	1: C	1:  D
2: C	(3, 3)	(5, 0)
2: D	(0, 5)	(2, 2)

Player 1, and Player 2, can each choose C or D.  1 and 2's utility for the final outcome is given by the first and second number in the pair.  For reasons that will become apparent, "C" stands for "cooperate" and D stands for "defect".

Observe that a player in this game (regarding themselves as the first player) has this preference ordering over outcomes:  (D, C) > (C, C) > (D, D) > (C, D).

D, it would seem, dominates C:  If the other player chooses C, you prefer (D, C) to (C, C); and if the other player chooses D, you prefer (D, D) to (C, D).  So you wisely choose D, and as the payoff table is symmetric, the other player likewise chooses D.

If only you'd both been less wise!  You both prefer (C, C) to (D, D).  That is, you both prefer mutual cooperation to mutual defection.

The Prisoner's Dilemma is one of the great foundational issues in decision theory, and enormous volumes of material have been written about it.  Which makes it an audacious assertion of mine, that the usual way of visualizing the Prisoner's Dilemma has a severe flaw, at least if you happen to be human.

Can Counterfactuals Be True?

Followup to:  Probability is Subjectively Objective

The classic explanation of counterfactuals begins with this distinction:

1. If Lee Harvey Oswald didn't shoot John F. Kennedy, then someone else did.
2. If Lee Harvey Oswald hadn't shot John F. Kennedy, someone else would have.

In ordinary usage we would agree with the first statement, but not the second (I hope).

If, somehow, we learn the definite fact that Oswald did not shoot Kennedy, then someone else must have done so, since Kennedy was in fact shot.

But if we went back in time and removed Oswald, while leaving everything else the same, then - unless you believe there was a conspiracy - there's no particular reason to believe Kennedy would be shot:

We start by imagining the same historical situation that existed in 1963 - by a further act of imagination, we remove Oswald from our vision - we run forward the laws that we think govern the world - visualize Kennedy parading through in his limousine - and find that, in our imagination, no one shoots Kennedy.

It's an interesting question whether counterfactuals can be true or false.  We never get to experience them directly.

When (Not) To Use Probabilities

Followup to:  Should We Ban Physics?

It may come as a surprise to some readers of this blog, that I do not always advocate using probabilities.

Or rather, I don't always advocate that human beings, trying to solve their problems, should try to make up verbal probabilities, and then apply the laws of probability theory or decision theory to whatever number they just made up, and then use the result as their final belief or decision.

The laws of probability are laws, not suggestions, but often the true Law is too difficult for us humans to compute.  If P != NP and the universe has no source of exponential computing power, then there are evidential updates too difficult for even a superintelligence to compute - even though the probabilities would be quite well-defined, if we could afford to calculate them.

So sometimes you don't apply probability theory.  Especially if you're human, and your brain has evolved with all sorts of useful algorithms for uncertain reasoning, that don't involve verbal probability assignments.

Not sure where a flying ball will land?  I don't advise trying to formulate a probability distribution over its landing spots, performing deliberate Bayesian updates on your glances at the ball, and calculating the expected utility of all possible strings of motor instructions to your muscles.

Probability is Subjectively Objective

Followup to:  Probability is in the Mind

"Reality is that which, when you stop believing in it, doesn't go away."
        -- Philip K. Dick

There are two kinds of Bayesians, allegedly.  Subjective Bayesians believe that "probabilities" are degrees of uncertainty existing in our minds; if you are uncertain about a phenomenon, that is a fact about your state of mind, not a property of the phenomenon itself; probability theory constrains the logical coherence of uncertain beliefs.  Then there are objective Bayesians, who... I'm not quite sure what it means to be an "objective Bayesian"; there are multiple definitions out there.  As best I can tell, an "objective Bayesian" is anyone who uses Bayesian methods and isn't a subjective Bayesian.

If I recall correctly, E. T. Jaynes, master of the art, once described himself as a subjective-objective Bayesian.  Jaynes certainly believed very firmly that probability was in the mind; Jaynes was the one who coined the term Mind Projection Fallacy.  But Jaynes also didn't think that this implied a license to make up whatever priors you liked.  There was only one correct prior distribution to use, given your state of partial information at the start of the problem.

How can something be in the mind, yet still be objective?

Where Recursive Justification Hits Bottom

Followup to:  No Universally Compelling Arguments, Passing the Recursive Buck, Wrong Questions, A Priori

Why do I believe that the Sun will rise tomorrow?

Because I've seen the Sun rise on thousands of previous days.

Ah... but why do I believe the future will be like the past?

Even if I go past the mere surface observation of the Sun rising, to the apparently universal and exceptionless laws of gravitation and nuclear physics, then I am still left with the question:  "Why do I believe this will also be true tomorrow?"

I could appeal to Occam's Razor, the principle of using the simplest theory that fits the facts... but why believe in Occam's Razor?  Because it's been successful on past problems?  But who says that this means Occam's Razor will work tomorrow?

And lo, the one said:

"Science also depends on unjustified assumptions.  Thus science is ultimately based on faith, so don't you criticize me for believing in [silly-belief-#238721]."

All Hail Info Theory

Hard questions are often hard because different ways to think about them conflict.  When each way seems to have strong support, we are reluctant to choose.  But if we cannot avoid the conflict, choose we must.  For example, last October I wrote:

Our standard ("Bayesian") formal theories of information and probability ... are by far the main formal approaches to such issues in physics, economics, computer science, statistics, and philosophy.  ... There are, however, a number of claimed exceptions, cases where many people think certain beliefs are justified even though they seem contrary to this standard framework. ... I am ... tempted to reject all claimed exceptions, but that wouldn't be fair.  So I'm instead raising the issue and offering a quick survey of claimed exceptions. ... The following do not seem to be exceptions: Indexicals ... Logical Implications ... Here are possible exceptions:  Math and Concept Axioms ... Basic Moral Claims ... Consciousness ... The Real World ... Real Stuff ... [I could have added religious beliefs to this list.]

Actually, in all these cases it seems it is standard info theory (i.e., info is whatever excludes possibilities) alone that seems to conflict with something else - probability theory is irrelevant.  And it seems to me that: Nothing that seems to conflict with standard info theory is remotely as well established as it is.  So when there is a conflict, info theory must just win.  (More are willing to challenge standard "Bayesian" probability theory - e.g., see Andrew Gelman, Scott Aaronson.) 

Class Project

Followup to:  The Failures of Eld Science, Einstein's Superpowers

"Do as well as Einstein?" Jeffreyssai said, incredulously.  "Just as well as Einstein?  Albert Einstein was a great scientist of his era, but that was his era, not this one!  Einstein did not comprehend the Bayesian methods; he lived before the cognitive biases were discovered; he had no scientific grasp of his own thought processes.  Einstein spoke nonsense of an impersonal God - which tells you how well he understood the rhythm of reason, to discard it outside his own field! He was too caught up in the drama of rejecting his era's quantum mechanics to actually fix it.  And while I grant that Einstein reasoned cleanly in the matter of General Relativity - barring that matter of the cosmological constant - he took ten years to do it.  Too slow!"

"Too slow?" repeated Taji incredulously.

"Too slow!  If Einstein were in this classroom now, rather than Earth of the negative first century, I would rap his knuckles!  You will not try to do as well as Einstein!  You will aspire to do BETTER than Einstein or you may as well not bother!"

Jeffreyssai shook his head.  "Well, I've given you enough hints.  It is time to test your skills.  Now, I know that the other beisutsukai don't think much of my class projects..."  Jeffreyssai paused significantly.

Brennan inwardly sighed.  He'd heard this line many times before, in the Bardic Conspiracy, the Competitive Conspiracy:  The other teachers think my assignments are too easy, you should be grateful, followed by some ridiculously difficult task -