Where Does Pascal’s Wager Fail?
The topic of Pascal’s wager has been mentioned several times before on Overcoming Bias, most notably in Eliezer’s post on Pascal’s mugging. I’m interested in discussing the question with specific reference to its original context: religion. My assumption is that almost all readers agree that the wager fails in this context — but where exactly?
One common objection is the many gods argument: While it’s true you might be punished eternally (or, if you like, for 3^^^^3 years) if and only if you don’t follow, say, Christianity, it’s possible to imagine other scenarios where you would be punished if and only if you do follow Christianity; thus, it’s claimed, the different possibilities cancel each other out. In responding to the Pascal’s-mugging post, Michael Vassar suggested that we should have Equal priors due to complexity, equal posteriors due to lack of entanglement between claims and facts. But are the priors really equal? Intuitively, the anti-Christian God should take more bits to describe, since that hypothesis requires stating the entire concept of Christianity and then a little extra. I don’t know if that’s the case, but my point is it’s not obvious that, bit for bit, the hypotheses are identical in Kolmogorov complexity. Moreover, the set of relevant hypotheses is bigger than these two: There are tons of hypotheses according to which whether you follow Christianity will make a difference as to whether you suffer for 3^^^^3 years, and I’m not convinced that they all exactly cancel out in prior probability.
Moreover, is there really no entanglement? Is the probability of observing the world we do exactly the same given Christianity as given anti-Christianity? Is the probability, given Christianity, that billions of people would be persuaded of the truth of the Christian God’s message exactly the same as the probability, given anti-Christianity, that billions of people would be fooled into believing the Christian God’s message? Or for that matter, are the probabilities that billions of people will follow non-Christian religions equal under the two scenarios? And so on. There seems to be just too much data in the world for our probabilities to remain symmetric.
This relates to a second complaint about the wager: With vast amounts of data to process and an enormous space of possible religious hypotheses to search, Pascal’s wager (which is just an optimization problem) is computationally infeasible, especially for human minds. This is true, but even if we can’t find the global optimum (if one exists?), I don’t see why we shouldn’t make what local improvements we can, given our limited knowledge, processing ability, and creativity in specifying hypotheses. Just by considering a few basic factual predictions that various religions make, for example, it ought to be possible to separate hypotheses of similar prior probability by many orders of magnitude in their posteriors. We could make some progress on these back-of-the-envelope calculations even without having a full Solomonoff-inducting AI (though the latter would indeed be extraordinarily helpful).
In view of the high uncertainty surrounding the question of which religion (possibly including atheism) to choose, maybe it would be best to avoid making a commitment now, since you might learn more as time goes on that would affect your choice. Moreover, there’s a small chance that in trying to adhere to the commands of a particular religion and in surrounding yourself with fellow believers, you might blunt your ability to think rationally. This argument is fine as far as it goes (though you should also consider your probability of dying before you finally do make up your mind), but then why not spend considerable effort doing further research on the question of which religion to follow? The expected value of additional information would seem to be extraordinarily high.
You might reply that the problem of which religion to follow is overly narrow: There are lots of other projects to work on, perhaps involving more probable scenarios than does Pascal’s wager. For instance, maybe you’re aiming for physical immortality via ordinary materialist means and intend to spend all your time researching how best to stay alive until significant anti-ageing technologies kick in. Fair enough, but what if — as is true in my case — you’re more concerned about avoiding eternal suffering, rather than achieving eternal blissful life? Are there secular scenarios that would require you not to consider Pascal’s wager in the religious case in order to prevent yourself from experiencing massive amounts of suffering?
Finally, some might object to using an unbounded utility function because it leads to mathematical difficulties. I admit that I don’t like the idea of bounding utility functions, but even if we do that, can we not take the bounds big enough that we still allow speculative Pascalian scenarios to dominate over more minor, worldly considerations?

Yvain has come closest to the truth here. As he states, the objections people make to the wager are ad hoc; they would reject it even if all the objections were known to be false.
Why is this? As I've stated before, human beings naturally have a bounded utility function, and anyone who decides to act as though he had an unbounded utility function, is deciding to act like a fanatic. With this bounded utility function, there is little expected value from anything with a sufficiently low probability. But if someone were really willing to act as though his utility function were unbounded, he would become a fanatic... and he would accept the wager.
I find all of the standard tricks used against Pascal's Wager intellectually unsatisfying because none of them are at the root of my failure to accept it. Yes, it might be a good point that there could be an "atheist God" who punishes anyone who accepts Pascal's Wager. But even if a super-intelligent source whom I trusted absolutely informed me that there was definitely either the Catholic God or no god at all, I feel like I would still feel like Pascal's Wager was a bad deal. So it would be dishonest of me to say that the possibility of an atheist god "solves" Pascal's Wager.
The same thing is true for a lot of the other solutions proposed. Even if this super-intelligent source assured me that yes, if there is a God He will let people into Heaven even if their faith is only based on Pascal's Wager, that if there is a God He will not punish you for your cynical attraction to incentives, and so on, and re-emphasized that it was DEFINITELY either the Catholic God or nothing, I still wouldn't happily become a Catholic.
Whatever the solution, I think it's probably the same for Pascal's Wager, Pascal's Mugging, and the Egyptian mummy problem I mentioned last month. Right now, my best guess for that solution is that there are two different answers to two different questions:
Why do we believe Pascal's Wager is wrong? Scope insensitivity. Eternity in Hell doesn't sound that much worse, to our brains, than a hundred years in Hell, and we quite rightly wouldn't accept Pascal's Wager to avoid a hundred years in Hell. Pascal's Mugger killing 3^^^3 people doesn't sound too much worse than him killing 3,333 people, and we quite rightly wouldn't give him a dollar to get that low a probability of killing 3,333 people.
Why is Pascal's Wager wrong? From an expected utility point of view, it's not. In any particular world, not accepting Pascal's Wager has a 99.999...% chance of leading to a higher payoff. But averaged over very large numbers of possible worlds, accepting Pascal's Wager or Pascal's Mugging will have a higher payoff, because of that infinity going into the averages. It's too bad that doing the rational thing leads to a lower payoff in most cases, but as everyone who's bought fire insurance and not had their house catch on fire knows, sometimes that happens.
I realize that this position commits me, so far as I am rational, to becoming a theist. But my position that other people are exactly equal in moral value to myself commits me, so far as I am rational, to giving almost all my salary to starving Africans who would get a higher marginal value from it than I do, and I don't do that either.