Even if I know you're coming, replying here will let others know who's coming.

Nonetheless, I would like to present some of my motivations on Newcomb's Problem – the reasons I felt impelled to seek a new theory – because they illustrate my source-attitudes toward rationality. Even if I can't present the theory that these motivations motivate…

First, foremost, fundamentally, above all else:

Rational agents should WIN.

As I just commented on another thread, this is faith in rationality, which is an oxymoron.

It isn't obvious whether there is a rational winning approach to Newcomb's problem. But here's a similar, simpler problem that billions of people have believed was real, which I'll call Augustine's Paradox ("Lord, make me chaste – but not yet!")

All conservative variants of Christianity teach, in one way or another, that your eternal fate depends on your state in the last moment of your life. If you live a nearly-flawless Christian life, but have a sinful thought ten minutes before dying and the priest has already left, you go to Hell. If you are sinful all your life but repent in your final minute, you go to Heaven.

The optimal self-interested strategy is to act selfishly all your life, and then repent at the final moment. But if you repent as part of a plan, it won't work; you'll go to Hell anyway. The optimal strategy is to be selfish all your life, without intending to repent, and then repent in your final moments and truly mean it.

I don't think there's any rational winning strategy here. Yet the purely emotional strategy of fear plus an irrationally large devaluation of the future wins.

I'm not entirely happy with this, because the problem assumes that God cares not just about what you do, but also why you do it, and sends you to Hell if you adopt a strategy for the purpose of winning.

We could say that any general strategy that doesn't apply only to the paradox itself is admissible. For instance, assume God allows you to adopt a self-identity function that discounts your identification with your future self at such a rate that repentance becomes rational only a few hours before your death, even if Augustine's paradox was part of your reason. The problem with that strategy is that your life overall would probably not be very winning.

But since the paradox as originally described, including God's caring about your motives, is a problem that billions of people have believed, we can't write it off and say "That's not a fair problem". As Eliezer has said, Nature doesn't care if problems are fair. If rationality is always the winning strategy, we must allow all possible Natures; and God is possible.

(Incidentally, googling turns up Augustine's paradox of time, Augustine's paradox of teaching, Augustine's paradox of humility, Augustine's paradox of memory and learning, and Augustine's paradox of creation.)

(I was already amused by the idea of closing Guantanamo because people there carried out policies decided on in Washington DC.  The logic could be that guilt adhered to the place itself; or that guilt could be made to adhere to it and then be done away with, as with a scape-goat.  If it worked with Jesus, why not with Guantanamo?)

But the idea that "being the right thing to do" is a reason rather than a conclusion is more intriguing.  Is this just circular logic?  I don’t think so.

Recall the Pope’s statements about reason vs. faith.  In this view, morality is associated with faith, which is contrasted with reason.  That’s the worldview America grew up with.

Take away the faith.  (This is NPR, after all.)  What happens?  After thousands of years of being attached to faith, does morality attach itself, in the minds of the public, to reason?  Or does it just become detached?  I think that the latter model explains the speaker’s statements: "The right thing" is something you just know – a support, not a conclusion.

A recent Science article, summarized here, reports on this as an instance of a simple yet subtle bias: When given information, people assume that the effects relevant to them scale linearly with the measurement scale used. In this instance, it’s miles per gallon.

If this were wartime, and you were rationed 10 gallons per week, the measurement of interest to you in evaluating a car’s mileage might be the number of different places you could visit once a week with that car.  Then the relevant statistic would be (miles/gallon)².  But since we aren’t rationing gas, a better measurement is gallons per mile, which can be translated into dollars and environmental impact per mile.

When people are given figures in miles per gallon, they usually think that the answer to the above question is 30 mpg.  "Sixty percent of participants ordered the pairs according to linear improvement and 1% according to actual improvement. A third strategy, proportional improvement, was used by 10% of participants."  (The proportional strategy says that the answer is 40 mpg.)

People get the right answer when you rephrase the question in units that scale linearly with the effect.  Try this:  Your first car could go 100 miles on 10 gallons of gas.  Your second car could go 100 miles on 5 gallons of gas.  Your third car needs to go 100 miles on… 0 gallons of gas.  So it needs to get infinite mpg, to match the improvement in going from 10 to 20 mpg.

(Incidentally, there probably is no viable distinction between cognitive structure and content.)

This statement is true, in that there is probably no distinction that I can write, that Jed can’t come up with a counter-example to.  Much as I can’t write a definition of "game" that Wittgenstein couldn’t come up with a counter-example to.

But the statement was used to imply that distinguishing AI architectures by reliance on content vs. learning is nonsensical.  If that were so, knowledgeable people would be confused when Eliezer (or Lenat) says Cyc emphasizes content more than other architectures do.  They aren’t.

Some more-popular false false dichotomies:

• Nature vs. nurture (e.g., genetic or instinctual vs. learned behavior):  We’re told that there’s no true distinction between them, since "nurture" can only occur when expected by "nature".  I like Paul Bloom’s reply (paraphrasing), "There’s something wrong with a theory of mind that says that a knee reflex and word learning are the same sort of thing."
• Race:  We’re told that race is a "social construct" because, for any particular genetic criteria you set to determine who is in a race, someone can be found who looks to us like they belong to that race, yet doesn’t satisfy your criteria.
• Gender:  There are people naturally having characteristics of both sexes; people whose phenotypic gender is different from their genotypic gender; and people who’ve had sex-change operations.  Therefore, there is no gender.

You probably knew where I was going with this when you saw the Wittgenstein reference.  Every word in our languages breaks down when you apply enough pressure to it.  A word encodes a statistical regularity.  Applicability in all cases is not required.  Forbid us from using words that aren’t precise, and we’d be unable to talk at all.

The number of times that a single vote has affected the outcome of a US presidential election is, so far, zero.

In order for voting to be rational, the expected benefit to you from your vote having an effect on the outcome, must be greater than the expected cost of you dying in an auto accident on your way to vote.

The traffic accident study covers only 32 years; but we have over 200 years of data on individual votes not swinging an election.  Over time, it has become much less likely for one person’s vote to swing an election due to population increase.  I will approximate this effect by saying that 210 years of one vote not swinging an election is similar to 1000 years of one vote not swinging an election at current population levels.  That’s a sloppy off-the-cuff guess at how the population changes affect the probabilities.

So, the odds of your dying in a traffic accident on your way to vote would at first seem to be 24 * (1000/4) = 6000 times the odds of your vote changing the outcome of the election.  (Probably much higher. Those are the odds they would be if one person’s vote had swung the election once.)  The odds of your being disabled in a traffic accident on your way to vote would, similarly, seem to be 800*(1000/4) = 200,000 times higher than the odds of your vote swinging the election.

But they aren’t.  Another recent study by 3 people (including Andrew Gelman, one of Overcoming Bias’ readers) (pdf here), using a detailed model, estimated that the odds of one person’s vote swinging the presidential election in 2008 were, on average, one in 60 million.  So why is it that, with more than 120 million people voting this year, no one came close to having their vote swing the election?

The problem is if your vote swung the vote in your state, the vote of half the other voters in your state also swung the vote.  The number of trials here is the number of states times two (one Republican and one Democrat trial per state).  So what that figure really means is that the "average" state has a 1 in 30 million chance of swinging the vote, and you have a 50% chance of being on the winning side in that case.

(To fix my first estimate, we need to say that the figures above indicate that a given party in a given state is unlikely to have more than a 1 in (100*1000/4) = 1 in 25,000 chance of swinging the election.  This then bounds the odds of any one person swinging the election to be no more than 1 in 2,500,000 on average.)

Swinging the vote is a "black swan" event that is highly unlikely.  If it happens for a state one time in 30 million, it should happen about once every 2,400,000 years.  But when it does, millions of people can claim their vote swung the election; the "expected number" of voters who swing the vote per election is actually two.

This is a black swan:  An event that occurs so infrequently that it has an expected number of occurrences in history near zero; yet of such magnitude that its expected impact is considerable.  (And the "expected number per event" is a number that can never happen.)  Relying on history to predict the odds of such an event, as I did above, mislead you.

Back to our practical application.  The chance of your dying in a car crash on the way to vote is really only about ten times as great as the chance of your vote altering the outcome of the election.  Less, if our models of traffic indicate that most of the excess deaths were among people not on their way to vote.  But you’re not going to get that figure low enough for it to be rational for you to vote in a presidential election, unless you’re willing to die to get your candidate elected.  (Or unless, like me, your voting place is within walking distance.)

Rationally, people should be much more interested in local elections, which they have a much greater chance of affecting.

What about the opportunity cost of voting?

The 2008 presidential election cost about $1 billion, spent contending over roughly 127 million * .2 = 24 million undecided voters.  One vote thus cost about $40.  If you make more than $20/hr (after taxes), you would be better off giving $40 to the campaign, than standing in line to vote for 2 hours.

(Unless you like standing in line better than working.  It could be a good way to meet people.)

"But wait," you say.  "You’ve only considered the cost to yourself.  I’m a good patriot!  I consider value to my country as party of my utility function!"

Well, now that we’ve converted our voting decision into dollar terms, we can say that you might vote if you expect electing your candidate, vs. the other candidate, to be worth at least $2,400,000,000 to your country.  Otherwise, you should stay home and give $40 to the US Treasury by burning 2 $20 bills.

George W. Bush’s presidency may have cost the US 10^3 to 10^4 times that figure.  Let’s say that having a better president is worth $10 trillion to the US.  (For comparison, US GDP is about $14 trillion/year.)  Let’s suppose that people who voted, did so because they were good patriots, and put the benefit to their country above the risk to their lives.  That is, the expected gain from having a better president due to their vote ($10 trillion, divided by 60 million) outweighs the personal cost (value of own life, divided by 6 million).

This means that our rational but patriotic voters are willing to die to save their country $1 trillion.  That’s not so irrational.