Discussion about this post

User's avatar
Overcoming Bias Commenter's avatar

The issue I see here, both with the mugging situation and the voting / dedicating one's life to a cause is that in the above no mention is given to the number of times one will be able to make such a decision. Expected values really only make sense in aggregates; so in the mugging situation Pascal would need to be reasonably certain to be presented with similar choices about a quadrillion times before making the bet would be a good choice to make.In dedicating yourself to a cause however, the aggregate does not come from one individual being able to dedicate their life multiple times over, but from many individuals doing so; the last paragraph above aludes to this, but does not accurately characterize the group of potential actors. It is not just a group of 100 people contemplating the same project that are relevant to the calculation, but actually all people on the planet capable of carrying out the same activity who may at some point be exposed to and join the cause ( there is of course a probability of occurrence for this as well ).

Expand full comment
dmytryl's avatar

@dae236742d199236c0defc6f8379edda:disqus Stephen R. Diamond:

Well, I may be able to estimate how much of a wild guess the hypothesis is. I.e. if I am wildly guessing 9 digits number, that's one in a billion chance.

The problem with this is that you obtain an upper bound on probability, and the actual probability can be arbitrarily lower due to parts of the guess that you did not count.

With the pascal's mugging there is other aspect. A charity working on x-risk, or an approach to x-risk, may very easily be worse than just working and giving money to 1 randomly chosen person on this planet, or to random person with PhD in mathematics, or the like. The fact that someone tells you they are the best deal, is not necessarily *any* information that they are the best deal. If you have say 2..3% of psychopaths in society and several percent narcissists as well, and none of those folks want to work, it's clear that the people who can do something are grossly outnumbered by those who either have no moral qualms with telling what ever or have their self assessment hard wired to 'awesome'.

At this point it is not really about probability assessments even but about choosing effective strategies that elicit response. E.g. you can choose a strategy whereby the utility of some action would be positive for the real deal, but negative for the fake. You can require some definitely-non-bullshit achievements in mathematics or computer science. The real deal will have some from the time they were studying, or the time they were happily working on the AI being unaware of the risks. Really cheap to show. But it is not worth it doing such just for the sake of defrauding you - it is easier to e.g. increase reach and defraud the most gullible.

Expand full comment
80 more comments...

No posts