Discussion about this post

User's avatar
Peter Gerdes's avatar

I don't think it actually presents a challenge to standard deciscion theory -- only the assumption that what we call our values and beliefs actually correspond to the decision theory concepts.

I mean you could model us as actually having values that highly prize signalling we aren't weird or divergent from our community. And looking at people's willingness to bet on their supposed beliefs in high stakes situations suggests that in some sense we don't really believe them.

Though, we should keep in mind (especially in AI discussions as people often make this error) that beliefs/values aren't real things out there in the world and are instead just hueristic idealizations used to predict behavior within a certain range of cases.

I mean, this can trivially be seen to be true with extreme torture where people can be made to say things even knowing that long term it will result in worse pain just to make the immediate pain stop. Or in the fact that what people profess to believe in religious contexts about life after death doesn't actually impact behavior in the way one might assume.

Ultimately, if you cut open a person or computer you don't find beliefs or values just algorithms for behavior. It's just that within certain parameters we can use these theoretical ascriptions to make predictions about that behavior.

Expand full comment
Leo Abstract's avatar

Been loving these explorations, really thought provoking. My supposition is that we evolved without respect to explicit values or beliefs, in an environment in which the feedback loops for cultural change were fairly swift and the rate of drift fairly slow. We shouldn't expect ourselves to be able to deal with this well. Hopefully the cautionary tales that accompany the physical ruins of our civilization will at least be entertaining and well written. Another Homer awaits.

Expand full comment
15 more comments...

No posts