The Modern Mind's Fatal Flaw
Here’s a quick history of minds. First, there were no minds, just stupid stuff undergoing simple local interactions. Then appeared the first proto-mind: Darwinian evolution, via DNA. Under this, the universe made choices according to a purpose: adaption. But such choices were made locally, using little memory, models, or foresight. Even so, organisms changed to better achieve this adaption purpose.
This DNA-based selection evolved actual minds, which did have memory, models, and foresight. This allowed organism actions to match context in far more detail. The purposes that drove such minds were a mishmash of selected local proxies. And basic mind plans changed very slowly, as that was managed by DNA evolution.
DNA-based evolution eventually gave human minds the ability to copy others’ behaviors in great detail. Which then allowed culture-based Darwinian evolution, which induced a much faster evolution of behaviors. Human minds then rapidly invented language, tools, norms/law, the sacred, governance, property, trade, money, and many other powerful inventions we expect our descendants to long find valuable.
The invention of writing allowed humans to accumulate more complex shared world models, in part via shared critical editing of such models. And also more abstraction, and more complex machines and orgs.
In the last few centuries larger scale mixing via wider talk/travel/trade has caused faster innovation, rising wealth, and the modern mind, which used more education, abstraction, and systems of thought like science/engineering/accounting. These enabled the design of much more effective and complex machines and organizations.
This modern mind was finally able to see all this history that I’ve just described, and to notice big conflicts between the implicit goals of the Darwinian evolution that has ruled us, and the goals in our minds. The modern mind was also able to envision a future where descendants directly make machines to replace our DNA-constructed minds, and also to envision global governance directly replacing DNA and cultural evolution in setting the path and terms of change. We have wondered what goals such artificial minds, coordinated by global governance, might pursue, and set ambitions to set their goals. In part from concern for how they’d treat their still-surviving ancestors.
However, we modern minds have been getting ahead of ourselves. We embody a big flaw ready to take down not only take us, but also any artificial minds and global orgs we manage to make. We’ve been assuming that the goals in our heads are coherent and stable, and could serve as the basis for a functioning civilization of artificial minds and global governance. But in fact they are broken and fast decaying, and unable to support such a new civilization.
A few centuries ago the world had hundreds of thousands of peasant cultures, each often facing famine, war, and disease, with conservative cultures in a slowly changing world. Which made for a healthy evolution of cultures, even if it made for many miserable lives. But the modern world has been merging into a global monoculture that is rich, healthy, and peaceful, in much faster changing contexts, and celebrating cultural activism. This plausibly breaks the cultural evolution process, at least at the level of the evolution of cultures; evolution within cultures is still going great.
As a result, our key status markers, norms, and shared values have been becoming less adaptive, inducing falling fertility and more generally the fall of our shared world civilization. Which will then plausibly lead to the loss of many modern gains, including many aspects of modern minds. Even artificial minds inheriting our habits of cultural evolution, or slaved to match human culture, would suffer this fatal flaw.
We must learn to accept that in a world of Darwinian evolution, the only way to get anything you love to last a long time is to mix it into an adaptive culture. I can only think of three solutions, each quite hard to achieve: let capitalism run everything, govern small groups by number-go-up tied to adaption futures markets, or govern a big-enough group by number-go-up tied to futures market estimates re a civ goal in conflict w/ civ fall, like when 1M live in space. (Maybe assisted by futarchy.)
In addition, visions of global governance replacing Darwinian evolution neglect the fact that all structures, including organisms, species, machines, minds, and orgs, have so far consistently rotted and decayed. The only anti-decay force we’ve ever seen is Darwinian evolution, wherein new less rotted versions replace older more rotted versions. We don’t yet know how to make a global governance org that wouldn’t decay. And of course global governance might also not actually pursue the goals initially promised to outsiders as it was instituted.
The modern mind has pioneered some amazing things, but much of that is at risk, due to the modern mind’s fatal flaw of having broken the cultural evolution of cultures. Until we fix that, we won’t be able to achieve either a healthy new civilization of artificial minds, nor a stable global governance to regulate or replace Darwinian evolution. Let’s roll up our sleeves; there’s work to do.
Global governance does not exist and can not. The norm for global organisations is individual corruption, nepotism, and self-appointed banditry. Take the UN, the EU, the WHO among others. Achieve nothing, accountable to no one. A blind alley in evolution not an end point.
> We’ve been assuming that the goals in our heads are coherent and stable,
No, we don't have to just assume that. Through the magic of thinking about stuff, we can start with incoherent goals and modify them to become more coherent. We do this by noticing cognitive dissonance between our goals, and switching to goals that have less dissonance. Eventually, if we keep doing this, we converge on some goals that are coherent and stable, so that more thinking about them doesn't induce a preference for different goals. That's reflective equilibrium, in moral philosophy.
I feel like you're really undervaluing critical thinking. A lot of what you call adaptive would be (and has been) destroyed by it. Cultures that undervalue critical thinking, such as fundamentalist religious cultures, are willingly blinding themselves, and as a result they make a lot of bad decisions. They're like little brained organisms, and cultures that value critical thinking are big brained organisms. Little brained organisms can be successful by being r strategists. Big brained organisms can be successful by being K strategists. And it's the big brained organisms that end up with more control over the environment.