Discussion about this post

User's avatar
Jack's avatar

What is "rationality"? Rationality doesn't exist. Or rather, it only exists in domains like math and engineering where some problems are defined in tractable ways.

To apply "rationality" to any decision broader than this is a category error. The "rationalist ideologies" that purport to do so – Marxism, Libertarianism, Effective Altruism, and so on – are as likely to go astray as they are to generate anything of value. Worse still, they tend to ignore contradictory evidence that might cause them to self correct because, hey, you can't argue with rationality! We're above reproach!

Many experts spend their lives mulling over simplified models of irreducibly complex phenomena, and grow so enamored of those models that they fail to grasp the brittleness of their conclusions. Their conclusions aren't *wrong* – there is real value in expertise after all – they just aren't as right as they think they are. Average people often see this more clearly than experts do.

Expand full comment
Phil Getts's avatar

Re. "When it started in ’06, this blog was near the center of the origin of a “rationalist” movement wherein idealistic youths tried to adapt rational styles and methods. While these habits did often impress others, and bond this community together, they alas came to trust that their leaders had in fact achieved unusual rationality, and on that basis embraced many contrarian but not especially rational conclusions of those leaders."

I agree, and I think we can pin down more-precisely what happened. I think the main problem with the rationalist movement was that "rationalism" doesn't mean what people think it does. It doesn't mean thinking effectively, or making good predictions. It means taking geometry as a model for all human thought. This makes a lot of terribly wrong assumptions, like:

- all thoughts and reasons can be represented in human language

- every word has a single Platonic definition, which applies to all uses of that word, so we don't have to pay attention to how we're using each word /at the moment/

- every word has a single Platonic definition, so symbolic AI will work

- every sentence in English is either True or False

- reasoning should be deductive

- if you reason deductively, you can be 100% certain of your conclusions

The rationalist community didn't consciously believe any of these things except the symbolic AI one; but it didn't worry about them, either; at least not compared to the phenomenologists and subjectivists in the continental philosophy tradition. Instead of using their knowledge of Bayesian reasoning, iterative optimization, information theory, and statistics to show how to resolve continentalism's paradoxes, they used the fact that all continental philosophers are ignorant of these things to dismiss the valid points those philosophers had made about the failures of strict rationalistic thought. They allowed rationalism to leech into their thought from the universal background Rationalism of Western civilization, and fell into the usual bad habits of rationalism: overconfidence, moral certainty, perfectionism, groupthink, and cults. They thought they could solve all their epistemological problems by focusing on things like how to set their priors, and ignored questions like how words mean, what values are and how a mind, person, or group "has" them, and why LLMs don't use Bayesian neural networks.

That finally led many to the most-common terminal failure mode of rationalism: the kind of irrationalism you get when you know rationalism doesn't work, yet keep all of its wrong assumptions, and conclude that either the physical world isn't the real world, or else the world is broken on a metaphysical level. E.g., the post-rat Buddhism, mysticism, spiritualism, phenomenology, post-modernism that I've seen at Fluidity Forum and Vibecamp.

Expand full comment
28 more comments...

No posts

Ready for more?