Reason’s Overreach

Our most basic and ancient system for learning the consequences of our actions is natural selection. Compared to cultural natural selection, DNA natural selection learns (about adaptiveness) more slowly, as DNA changes more slowly. To learn faster, DNA selection created brains, to which cultural selection has added reason, science, and other tricks. But reason and brains only learn about a limited range of relevant action consequences.

The ability of brains using reason to figure out the consequences of actions is limited by the data available to them. Data on the consequences of actions are more available for more visible decisions that more people make more frequently, and less available for decisions that fewer people take less often. And in general, the adaptiveness of choices tends to be harder for brains and reason to see, compared to other familiar features and consequences of choices.

Furthermore, social groups often have game theoretic equilibria, and the “cultural” features of these groups which set these equilibria are not only hard for individuals to see, but also have consequences that are hard to see. To see the adaptive consequences of a particular equilibria-determining cultural feature, you’d need to see how changes to features result in changes to equilibria which result in adaptive consequence changes for that group. The larger is the group, the slower are feature changes, and the longer-term are consequence changes, the harder for individual brains and reason to see them.

Species, corporate cultures, and macro cultures all have (at least) two levels of evolution: inside, of features that are relatively easy to vary within such units, and between, of (cultural equilibria) features that are hard to vary within such units. An environment in which niches are smaller, making these units smaller, has worse (adaptive) innovation inside, as smaller units have fewer places inside for innovations to start, but better (adaptive) innovation between, as there are more units that try more candidates. And innovation is overall faster when units are smaller, suggesting between innovation matters more than inside innovation. We have good data showing this for species and corporate cultures, and I presume it also holds for macro cultures.

All of which leads me to this key conclusion: brains and reason are greatly limited in their ability to figure out the adaptive consequences of the cultures of larger social groups, especially the macro cultures of our largest groups. Such cultures are harder to see, their consequences take longer to play out and are also harder to see, and we just have far less data on such large things, even if we could see all of them. And over the last few centuries, the size of typical units has greatly increased, from peasant villages to nations to a world elite monoculture.

At many times and places in history, reason temporarily rose in status. For the case of Europe since ~1700, this rise has been unusually high and long lasting. Economic growth greatly accelerated, due to faster innovation often credited to reason, and mass education greatly raised the status of reasoners. The rise in the status of reason and innovation came at the expense of traditional authorities, who often resisted innovations, and who were often proven to be factually wrong.

During this period, wannabe innovators and reformers, and reasoners of many sorts, have used reason to suggest many changes. The rate of these suggested and realized changes has increased over time. And while initially most suggested changes were about specific tech and practices, over time we’ve come to see more proposed changes to basic institutions, practices, norms, and values of many sorts. Over time we’ve also raised the status of those we credit for innovations, and we’ve given our highest status to successful cultural activist who cause changes to our basic norms and values.

All of which might be fine if we were appropriately well-calibrated about the relative difficulty of these various innovation tasks. And made sensible tradeoffs re our cultural scales. But it seems to me that we have not done these. What we instead did was to vastly increase the scales of our macro cultures, ignoring the effects of that on the effectiveness of our cultural evolution process, and plausibly tipping that into a regime of maladaptive cultural drift. And we have eagerly advocated for and enacted proposals for cultural change on the basis of dramatically inadequate supporting data.

As I reviewed above, adaptiveness is hard to see, cultures and their consequences are hard to see, we have less data on the adaptiveness of features that are hard to vary within cultures, and less data on macro cultures which are larger, and we have moved to to max out the size of our macro cultures. All of which should make us quite hesitant to change key features of those macro cultures on the basis of reason. Yet we have not been remotely as hesitant as the considerations above recommend.

Plausible causes of this excess eagerness to change culture using reason include a failure to notice that reason does worse when it has far less data, an expectation that any failures could be noticed and corrected at modest costs, or, my best guess, a habit of using reason to estimate outcomes other than adaptiveness.

That is, people seem satisfied when reason suggests that a proposed reform would reduce suffering, increase happiness, etc., without wondering if it might have hidden costs to adaptiveness. People either assume that such changes typically increase adaptiveness, or that they don’t care much about adaptiveness, relative to such things.