"Change can be good when we sufficiently vet each change to verify that it makes us better. But much more change than that is very bad, though it may take a while for its harms to reveal themselves. Beware value change."
Hi Robin—big fan of your writing. I’ve been thinking about this alot and I disagree that “We roughly understand how more wealth, better science, faster communication, larger orgs, and more tolerance for experimentation, has allowed for sufficient selection among the many trials to support this faster rate of tech gains.”
I don’t think there’s anywhere near sufficient selection on the faster rate of tech—a single mutation happens and it spreads through the population almost instantaneously, unlike genetic selection, which must spread much slower, and is therefore subject to a much more stringent “pressure test.” This slow rate of change is also better at vetting longer term consequences and knock-on effects. We don’t do anything like this with tech. As Sarah Hill points out—we will likely look back on the pill one hundred years from now and be shocked at how cavalier we are with women’s hormones.
What’s more, tech change feeds back on value change. Every big leap in tech opens new pathways that cause us to change our evaluations and priorities. So if the fast rate of value change is an issue, then so is the rate of tech change.
I suspect we may have chosen a long, circuitous path that ultimately dead-ends the moment we used our first stone-tool. Relying on tech is an inherently unstable and risky long-term strategy, much like our reliance on endogenous Ascorbic acid—it anchors us to an external process that isn’t subject to the same rigorous, slow pressure of genetic selection and leaves us super vulnerable.
I meant that we have sufficient selection re the local short term effects, not the global long term effects. So far those other effects have on average been small enough to allow this neglect to be okay, but yes we have no guarantee that will continue.
When a behavior is the direct result of DNA selection pressures, that's called instinct. Instincts have to be simple (in comparison to cultural behaviors) because there is a limited amount of selection pressure - a limited pool of animals that can favor/disfavor a trait by reproducing or dying. That limited amount of pressure also has to be shared between selecting for instincts and selecting for physical characteristics. So, instincts can evolve only very slowly, and tend to be fairly simple.
The strength of humans is that we have lots of behaviors that are *not* the direct result of selection pressures. Instead, selection pressures have given us the traits of "learning from experience," "learning from others," "inventiveness," and "reasoning." We then apply these traits recursively to each other, resulting in behaviors far more complex than DNA selection pressure could directly produce. We leverage memetic selection, not just DNA selection. We always have done this; that's our success strategy as humans, in contrast to other animals that rely on DNA selection pressures to directly mold their behavior.
Memetic selection is not random drift, nor is it DNA selection. It's something else, something you need a different framework to understand. Memes rise and fall not because the people holding them live or die, but because the memes themselves are selected in the marketplace of ideas, based on coherence with other memes.
All memes are value-memes; they are all preferences about ways to think. There is no distinction to make, because what you think of as non-value-memes can be understood as value-memes from the right perspective. You yourself have alluded to this when you talked about the difficulty of distinguishing between values and beliefs in your "vote values, bet beliefs" credo. A belief is a preference about what patterns of thought to have.
And the way we select among these preferences is via coherence. If a person wants X and wants Y, but X conflicts with Y, then their preferences are incoherent, and as soon as they understand this incoherence, they have a drive to change their preferences and make them coherent.
For instance, a person may wish people to be healthy and well-educated, and also wish for a certain economic system, but the fact of the matter is that the economic system causes people to be unhealthy and ignorant. Once the person understands this, they will be driven to revise their preferences.
What about the "facts/values" distinction?!? A classification of all culturally-transmitted information as being "values" seems to be both unconventional and unhelpful.
It's helpful here because the same selection mechanisms apply to both; there is no special exception (such as Hanson is claiming) where memetic selection works on "facts" but not on "values." Memetic selection works in the same way on all memes, whether those memes are clothing fashions, scientific theories, religious traditions, legal systems, internet jokes, or moral norms. It's all about coherence.
A meme is adopted by a person if that person prefers to hold it, and what they prefer to hold is determined by the memes already in their head.
I think many readers are likely to choke on the claim that "all memes are value-memes" - and then not bother with the rest of your argument. A more conventional position is that memes vary in the extent to which acquiring them modifies the value system of the host. Religious claims about abortion being murder could potentially have an effect on host values. However, most memes probably have little effect on host values. If you ask someone for the time, and they respond then you have gained information from someone else - but it probably won't make much difference to your values and preferences.
Neither you nor I should care what is a "conventional" claim. That's not a rational basis for deciding to believe something.
If you ask someone for the time, you did so because you had some plan for what to do that depended on the time. Say, you wanted to go to the bank before it closes. Having learned the time, your preference for what activity to choose in the next hour changes. Perhaps you now prefer going to the bank immediately over having a coffee.
And of course, there is the preference that always applies to any belief, that now that you know it's 2:30, you prefer not to think it's 7:15.
It is veering off topic - but survival of memes is not really all about "coherence". Think about the Dawkins replicator trinity, for example. Fidelity, Fecundity, Longevity. Dawkins argues that these are all important properties - resisting mutation, making copies and persisting. Coherence alone seems like a rather simplistic paradigm.
Fidelity, fecundity, and longevity all strongly depend on coherence.
Fidelity: if a meme is coherent with other memes a person holds, then when the person hears the new meme, it will snap into their existing belief framework like a Lego piece. If a meme is incoherent with the other memes, then either the person will reject it entirely or will modify it (loss of fidelity) so that it fits better with their existing framework.
Fecundity: The more coherent a meme is with other memes person X holds, the more willing person X will be to share the meme with person Y, because people are willing to share what they strongly believe in. The more coherent a meme is with other memes person Y holds - if person X knows this - the more willing person X will be to share the meme with person Y, because they know it will be well-received.
Other factors do influence fecundity, such as the degree to which sharing the meme might increase the status of person X or better signal their group affiliation. Coherence is required, though.
Longevity: The more coherent a meme is with existing memes, the longer you can remember it. Memory techniques work by associating a meme with other memorable features.
Re: "It's something else, something you need a different framework to understand." Maybe, but if the evolutionary theory is properly formulated than many of its tenets cover both the organic and cultural realms. Copying, selection, mutation and drift are basic foundational block of evolutionary theory that apply equally well to both the organic and cultural realms. That idea is part of what is often known as "Universal Darwinism". Yes, cultural evolution features more inductive inference and intelligent design, but that just means that these are not such foundational concepts.
A different framework from DNA selection. Mutation, copying, and selection still apply to memes but it is necessary to understand that this mostly is not about people or groups of people surviving or dying, it's about ideas taking root or not taking root in people's minds.
The distinction between proximate and ultimate values seem to be missing from this post. I think it's relevant - and that an appreciation of it makes the situation less fragile and dangerous. Our ultimate values are things that tend to be hard-wired into our biology - warmth satiation, lack of pain - and so on. Proximate values are things that help us attain our ultimate values in the long term. They can include a preference for red berries over blue berries or worship of the moon god vs the sun god. Many proximate values are possible to derive from ultimate values (and/or the environment) via individual learning. So, for example, if your parents forgot to tell you which god to worship, you can often figure that out from local clues. Ultimate values are often genetic and are stored in DNA - while proximate values are more likely to be culturally-transmitted.
If you lump all of our values together then the result does look like a complex system that could easily get broken by random mutations. However, if you mostly concern yourself with ultimate values, then things don't seem so bad. These are a much smaller set which requires much less selection to maintain. Plus they don't require much in the way of high-level selection to maintain. When they break, it is typically fairly catastrophic to the individual.
This is a "jukebox" theory of values: someone picks out a record (ultimate values) and they then get a lot of details about the individual notes in the songs "free of charge" (proximate values).
This is not to say that cultural transmission of proximate values is completely useless. Deriving them from ultimate values (plus the environmental state) can take time and effort. However it does make them less sensitive to corruption via cultural mutations. They can be derived again from ultimate values - or the rest of the environment - with some effort. Or maybe they don't really matter much - like which side of the road to drive on. If that preference got totally lost somehow, it could just be reinvented again later.
Even if you were right - and your argument is full of holes I've pointed out in the past - the only "bad" outcome would be an eventual decrease in the population (only starting after 2080), which you claim would be compensated for by the Amish anyway, an outcome you more and more have approved of. So why do you care? Even in the worst case you say it's self-correcting.
There are real and present things to worry about, like the unprecedented concentration of wealth among billionaires and giant corporations in the past 20-40 years, the unprecedented level of surveillance of the average citizen by governments and corporations, and the potential for AI to displace human workers. Society is going to change very drastically in the next 20 years as a result of these factors, becoming far more stratified and autocratic as power continues to migrate upward in the economic hierarchy.
For some reason probably best-known to Robin, he is not sold on short timelines for machine intelligence. That removes several of the the most immedate sources of concern and makes other things seem more important by comparison.
Human fertility decline also features in many scenarios involving machine intelligence. Humans may get sucked into the matrix, uploaded into virtual worlds, get immersed in video games, or find it difficult to make a living. Fewer real humans babies seems to be a common feature across many different future scenarios.
Well, yes, AGI is likely to reduce the human population, by displacing human workers if nothing else. However, the solutions to this problem are very different from the paleo-conservative ideas Robin is talking about. UBI, vat-grown babies, legislative mandates that certain jobs must be done by humans, direct subsidies to human workers, taxes on AI.
Your writings on cultural drift remind me of God Emperor of Dune somewhat, he saw that humanity was headed towards extinction and made a lot of tough decisions to prevent it creating a kind of large scale dogma among humans to resist future tyranny. (Living for thousands of years while maintaining a stranglehold on power also helped). Maybe that's the answer, foster dogmatic beliefs around the importance of vetting change to be passed down through the generations.
you say to judge each change, but many things only have an effect when paired with other things, sometimes a great many other things.
Free speech only works if you also have freedom of religion--that's why the first amendment includes many subjects. Or, the fourth and fifth amendment need each other--that's why the bill of rights includes many amendments.
This is the entire point of intersectionalism--if you only ban race-discrimination, then black-women will still be discriminated against, and if you only ban sex-discrimination, then gay-blacks or gay-women will be discriminated against.
It seems to me we have multiple learning loops, and the current ones are largely part of reaching equilibrium from the shock of Christian values, themselves a mutation from Judiac values.
In regards to activists - what causes certain activist values to be pushed, and what causes them to take?
I think it's ideas and values already in the pool, and it's part of reaching equilibrium.
Excellent, well-explained conclusion:
"Change can be good when we sufficiently vet each change to verify that it makes us better. But much more change than that is very bad, though it may take a while for its harms to reveal themselves. Beware value change."
Hi Robin—big fan of your writing. I’ve been thinking about this alot and I disagree that “We roughly understand how more wealth, better science, faster communication, larger orgs, and more tolerance for experimentation, has allowed for sufficient selection among the many trials to support this faster rate of tech gains.”
I don’t think there’s anywhere near sufficient selection on the faster rate of tech—a single mutation happens and it spreads through the population almost instantaneously, unlike genetic selection, which must spread much slower, and is therefore subject to a much more stringent “pressure test.” This slow rate of change is also better at vetting longer term consequences and knock-on effects. We don’t do anything like this with tech. As Sarah Hill points out—we will likely look back on the pill one hundred years from now and be shocked at how cavalier we are with women’s hormones.
What’s more, tech change feeds back on value change. Every big leap in tech opens new pathways that cause us to change our evaluations and priorities. So if the fast rate of value change is an issue, then so is the rate of tech change.
I suspect we may have chosen a long, circuitous path that ultimately dead-ends the moment we used our first stone-tool. Relying on tech is an inherently unstable and risky long-term strategy, much like our reliance on endogenous Ascorbic acid—it anchors us to an external process that isn’t subject to the same rigorous, slow pressure of genetic selection and leaves us super vulnerable.
Would be interested in your thoughts on this.
I meant that we have sufficient selection re the local short term effects, not the global long term effects. So far those other effects have on average been small enough to allow this neglect to be okay, but yes we have no guarantee that will continue.
When a behavior is the direct result of DNA selection pressures, that's called instinct. Instincts have to be simple (in comparison to cultural behaviors) because there is a limited amount of selection pressure - a limited pool of animals that can favor/disfavor a trait by reproducing or dying. That limited amount of pressure also has to be shared between selecting for instincts and selecting for physical characteristics. So, instincts can evolve only very slowly, and tend to be fairly simple.
The strength of humans is that we have lots of behaviors that are *not* the direct result of selection pressures. Instead, selection pressures have given us the traits of "learning from experience," "learning from others," "inventiveness," and "reasoning." We then apply these traits recursively to each other, resulting in behaviors far more complex than DNA selection pressure could directly produce. We leverage memetic selection, not just DNA selection. We always have done this; that's our success strategy as humans, in contrast to other animals that rely on DNA selection pressures to directly mold their behavior.
Memetic selection is not random drift, nor is it DNA selection. It's something else, something you need a different framework to understand. Memes rise and fall not because the people holding them live or die, but because the memes themselves are selected in the marketplace of ideas, based on coherence with other memes.
It is much harder to select ways to pick value-memes on the fly, compared to other memes. Coherence has limited value there.
All memes are value-memes; they are all preferences about ways to think. There is no distinction to make, because what you think of as non-value-memes can be understood as value-memes from the right perspective. You yourself have alluded to this when you talked about the difficulty of distinguishing between values and beliefs in your "vote values, bet beliefs" credo. A belief is a preference about what patterns of thought to have.
And the way we select among these preferences is via coherence. If a person wants X and wants Y, but X conflicts with Y, then their preferences are incoherent, and as soon as they understand this incoherence, they have a drive to change their preferences and make them coherent.
For instance, a person may wish people to be healthy and well-educated, and also wish for a certain economic system, but the fact of the matter is that the economic system causes people to be unhealthy and ignorant. Once the person understands this, they will be driven to revise their preferences.
What about the "facts/values" distinction?!? A classification of all culturally-transmitted information as being "values" seems to be both unconventional and unhelpful.
It's helpful here because the same selection mechanisms apply to both; there is no special exception (such as Hanson is claiming) where memetic selection works on "facts" but not on "values." Memetic selection works in the same way on all memes, whether those memes are clothing fashions, scientific theories, religious traditions, legal systems, internet jokes, or moral norms. It's all about coherence.
A meme is adopted by a person if that person prefers to hold it, and what they prefer to hold is determined by the memes already in their head.
I think many readers are likely to choke on the claim that "all memes are value-memes" - and then not bother with the rest of your argument. A more conventional position is that memes vary in the extent to which acquiring them modifies the value system of the host. Religious claims about abortion being murder could potentially have an effect on host values. However, most memes probably have little effect on host values. If you ask someone for the time, and they respond then you have gained information from someone else - but it probably won't make much difference to your values and preferences.
Neither you nor I should care what is a "conventional" claim. That's not a rational basis for deciding to believe something.
If you ask someone for the time, you did so because you had some plan for what to do that depended on the time. Say, you wanted to go to the bank before it closes. Having learned the time, your preference for what activity to choose in the next hour changes. Perhaps you now prefer going to the bank immediately over having a coffee.
And of course, there is the preference that always applies to any belief, that now that you know it's 2:30, you prefer not to think it's 7:15.
It is veering off topic - but survival of memes is not really all about "coherence". Think about the Dawkins replicator trinity, for example. Fidelity, Fecundity, Longevity. Dawkins argues that these are all important properties - resisting mutation, making copies and persisting. Coherence alone seems like a rather simplistic paradigm.
Fidelity, fecundity, and longevity all strongly depend on coherence.
Fidelity: if a meme is coherent with other memes a person holds, then when the person hears the new meme, it will snap into their existing belief framework like a Lego piece. If a meme is incoherent with the other memes, then either the person will reject it entirely or will modify it (loss of fidelity) so that it fits better with their existing framework.
Fecundity: The more coherent a meme is with other memes person X holds, the more willing person X will be to share the meme with person Y, because people are willing to share what they strongly believe in. The more coherent a meme is with other memes person Y holds - if person X knows this - the more willing person X will be to share the meme with person Y, because they know it will be well-received.
Other factors do influence fecundity, such as the degree to which sharing the meme might increase the status of person X or better signal their group affiliation. Coherence is required, though.
Longevity: The more coherent a meme is with existing memes, the longer you can remember it. Memory techniques work by associating a meme with other memorable features.
Re: "It's something else, something you need a different framework to understand." Maybe, but if the evolutionary theory is properly formulated than many of its tenets cover both the organic and cultural realms. Copying, selection, mutation and drift are basic foundational block of evolutionary theory that apply equally well to both the organic and cultural realms. That idea is part of what is often known as "Universal Darwinism". Yes, cultural evolution features more inductive inference and intelligent design, but that just means that these are not such foundational concepts.
A different framework from DNA selection. Mutation, copying, and selection still apply to memes but it is necessary to understand that this mostly is not about people or groups of people surviving or dying, it's about ideas taking root or not taking root in people's minds.
The distinction between proximate and ultimate values seem to be missing from this post. I think it's relevant - and that an appreciation of it makes the situation less fragile and dangerous. Our ultimate values are things that tend to be hard-wired into our biology - warmth satiation, lack of pain - and so on. Proximate values are things that help us attain our ultimate values in the long term. They can include a preference for red berries over blue berries or worship of the moon god vs the sun god. Many proximate values are possible to derive from ultimate values (and/or the environment) via individual learning. So, for example, if your parents forgot to tell you which god to worship, you can often figure that out from local clues. Ultimate values are often genetic and are stored in DNA - while proximate values are more likely to be culturally-transmitted.
If you lump all of our values together then the result does look like a complex system that could easily get broken by random mutations. However, if you mostly concern yourself with ultimate values, then things don't seem so bad. These are a much smaller set which requires much less selection to maintain. Plus they don't require much in the way of high-level selection to maintain. When they break, it is typically fairly catastrophic to the individual.
This is a "jukebox" theory of values: someone picks out a record (ultimate values) and they then get a lot of details about the individual notes in the songs "free of charge" (proximate values).
This is not to say that cultural transmission of proximate values is completely useless. Deriving them from ultimate values (plus the environmental state) can take time and effort. However it does make them less sensitive to corruption via cultural mutations. They can be derived again from ultimate values - or the rest of the environment - with some effort. Or maybe they don't really matter much - like which side of the road to drive on. If that preference got totally lost somehow, it could just be reinvented again later.
your recent posts on this topic really changed my mind.
Even if you were right - and your argument is full of holes I've pointed out in the past - the only "bad" outcome would be an eventual decrease in the population (only starting after 2080), which you claim would be compensated for by the Amish anyway, an outcome you more and more have approved of. So why do you care? Even in the worst case you say it's self-correcting.
There are real and present things to worry about, like the unprecedented concentration of wealth among billionaires and giant corporations in the past 20-40 years, the unprecedented level of surveillance of the average citizen by governments and corporations, and the potential for AI to displace human workers. Society is going to change very drastically in the next 20 years as a result of these factors, becoming far more stratified and autocratic as power continues to migrate upward in the economic hierarchy.
For some reason probably best-known to Robin, he is not sold on short timelines for machine intelligence. That removes several of the the most immedate sources of concern and makes other things seem more important by comparison.
Human fertility decline also features in many scenarios involving machine intelligence. Humans may get sucked into the matrix, uploaded into virtual worlds, get immersed in video games, or find it difficult to make a living. Fewer real humans babies seems to be a common feature across many different future scenarios.
Well, yes, AGI is likely to reduce the human population, by displacing human workers if nothing else. However, the solutions to this problem are very different from the paleo-conservative ideas Robin is talking about. UBI, vat-grown babies, legislative mandates that certain jobs must be done by humans, direct subsidies to human workers, taxes on AI.
Your writings on cultural drift remind me of God Emperor of Dune somewhat, he saw that humanity was headed towards extinction and made a lot of tough decisions to prevent it creating a kind of large scale dogma among humans to resist future tyranny. (Living for thousands of years while maintaining a stranglehold on power also helped). Maybe that's the answer, foster dogmatic beliefs around the importance of vetting change to be passed down through the generations.
you say to judge each change, but many things only have an effect when paired with other things, sometimes a great many other things.
Free speech only works if you also have freedom of religion--that's why the first amendment includes many subjects. Or, the fourth and fifth amendment need each other--that's why the bill of rights includes many amendments.
This is the entire point of intersectionalism--if you only ban race-discrimination, then black-women will still be discriminated against, and if you only ban sex-discrimination, then gay-blacks or gay-women will be discriminated against.
If combos is what matters, then combos must be judged.
How many centuries do you mean?
~2-3
Do you mean the Enlightenment, the Romantic Period--both? Can you be specific about which dysgenic cultural activists and their progeny you mean?
It seems to me we have multiple learning loops, and the current ones are largely part of reaching equilibrium from the shock of Christian values, themselves a mutation from Judiac values.
In regards to activists - what causes certain activist values to be pushed, and what causes them to take?
I think it's ideas and values already in the pool, and it's part of reaching equilibrium.