This is our monthly place to discuss relevant topics that have not appeared in recent posts.
“Pascal’s Mugging”–Its solution and, more importantly, why it can seem puzzling.
I don’t know whether the Pascal’s Mugging Paradox, if it can be called that, is even known outside OB/LW circles, but here it’s been regarded as a serious puzzle. I posted a near-mode rendition of the solution on LW. (http://lesswrong.com/r/discussion/lw/k9z/pascals_mugging_solved/) Here, it will be easier to reach the big picture.
I only happened on the solution; I had previously seen no reason to question the reasoning because it was easy to attribute the paradox to the wacky Salomonof’s induction hypothesis on which it’s based. But after solving the paradox by other avenues, Salamonof’s induction doesn’t seem like to bad a model for the relevant kind of simplicity, although I reject its general application.
Before cutting to the chase, I understand that Robin offered a solution to the paradox according to which asymmetrical effects on large numbers of people are “penalized.” I haven’t seen any paper on the subject, but if the summary is accurate, I would have to say that it misses the point. The paradox can be stated without multiplying entities. Anyway, once you see where the paradox goes wrong, all ad hoc solutions become unnecessary.
Here’s the solution. The paradox depends on the assumption that, since people sometimes tell the truth, we should assign some credence to the mugger’s catastrophic claims. As these claims become outlandish, we lose confidence in them, but they still impart some minute amount of credence, which, because of their huge magnitude, dominates the improbability of the harms.
But check your assumptions! The mugger’s claim doesn’t continue adding credence. The structure of the mugging puzzle provides the mugger with the incentive to maximize his prediction, and to the extent we believe he is maximizing, we have decreased reason to believe his prediction: decreasing to the point where it is more likely that the catastrophe will occur if you allow yourself to be mugged. A sufficiently strongly supported alternative hypothesis which contradicts truth telling, that is “maximizing,” is not only disqualified as evidence but becomes disconfirming evidence. (You would predict that a catastrophe claim would be reasonable. If it is unreasonable, it dominates, if it does, only up to where it isn’t so unreasonable that the total evidence for its contrary-that the catastrophe will occur if you don’t submit-isn’t greater than it will occur if you submit.)
I think the more important question is why does this puzzle seem more challenging than it is? I think this question provides a window to a human cognitive biases. We believe that that our statements increase the credibility our statements. This belief is basic to our signaling mentality.
It would be cognitively dissonant (http://disputedissues.blogspot.com/2012/10/uncomfortable-ideas-and-disfluent.html) for an advocate of measures claimed to reduce a risk of existential magnitude to think that his making these claims decreases the probability that they are true.
> I think the more important question is why does this puzzle seem more challenging than it is?
One possibility is that the puzzle is more challenging than you think 🙂
If the mugger threatens to torture a googolplex people, would any amount of evidence convince you that he’s telling the truth?
Actually, I kind of think folks just enjoy talking about googolplexes and matrix lords. (Or do you care to indicate how specifying the stakes actually changes the problem?)
Call their bluff: say that you are the secret overlord of The Matrix and will erase him from existence if he doesn’t walk away right now.
Another possibility is that as the magnitude of the threat increases the likelihood of its truthfulness decreases at a faster rate. 1/p(True) < O(magnitude)
This might be called the standard (insufficient) response: everyone takes my solution for being just this. The problem would be motivating it without violating the assumptions of salamanoff’s induction.
My solution–which is pretty obviously the solution–avoids having to compare rates of decline by the same move as is made in resolving the original Pascal’s Wager: invoking contradictory hypotheses. In Pascal’s wager, it’s that a different god, equally improbable, might have policies contradicting the imagined God. In Pascal’s “mugging,” when you have sufficiently strong reason to believe that the mugger is deceiving you, then you have less reason to believe his claim than its opposite.
In “solving” the puzzle, folks try to beat the math, which, of course, is impossible. All the chicanery lies (has to lie) in the assumption that claiming p is evidence of p, all else being equal–an empirical claim that’s undermined by the problem’s context.
Pascal’s mugger seemed problematic because folks have been insufficiently bayesian in understanding its terms.
ummm… will your solution stand if agent mugging you is an AI? (i.e not necessarily an agent that is constructed with a implicit goal to gain recources by cheating?)
And there is infinitesimal, but not zero probability that agent you identify as a person is actually a computer.
With high enough threat infinitesimality of this probability again becomes irrelevant and you *should* pay…
It just looks like I didn’t understand your proposition at all, and I have read your topic on LW before.
The solution doesn’t depend on discerning the deeper motives of the entity, only the likelihood that it is being untruthful (by maximizing).
I wonder what would happen to someone if you somehow suppressed their mental tendencies towards the Fundamental Attribution Error with technology. Would that “break” them as functional people, or would they be healthier for it since they would now have to properly weigh risks?
For clarification, if anyone else was confused: TheBrett is not suggesting that there’s a fundamental attribution error specific to technology. Rather, he’s suggesting using technology to eradicate the fundamental attribution error.
Thanks! I’ve edited my answer to notify people of that too.
How “magical” do you mean this technology to be? If you could take a magic pill that automatically corrected your attribution biases and did nothing else, it would seem considerable a benefit. The only loss might be that it makes hypocrisy harder, which might really be good as well.
However, if as Robin contends, the fundamental error of attribution is a feature of far-mode cognition, and correcting it meant damaging or eliminating far-mode capacities, the solution would be rather crippling. [I know there are some enemies of far-mode around who’d probably disagree.]
You say in your bio you studied by yourself at Standord, were you not enrolled and just used the libraries? I’m considering buying the 500 dollar access pass and studying their myself. Is this what you did or were you affiliated?
I had access via my working at NASA Ames nearby.
Tolstoy’s intuition about Near and Far modes. From War and Peace:
“In the past he had never been able to find that great inscrutable infinite something. He had only felt that it must exist somewhere and had looked for it. In everything near and comprehensible he had only what was limited, petty, commonplace, and senseless. He had equipped himself with a mental telescope and looked into remote space, where petty worldliness hiding itself in misty distance had seemed to him great and infinite merely because it was not clearly seen.”
Yes, the basic concepts are ancient.
It would have been better to encourage the poster rather than one-up him.
… be a charity angel.