19 Comments

Susan Blackmore is one of those who has looked into the effect of culture on the self. See articles like "Meme, Myself, I". Memes love to be considered part of your identity - since that way they qualify for special protection from competitors. So, people's identity tends to quickly become saturated with those memes that try to hook onto it. There's plenty of self-help advice out there related to what you can do about this situation.

Expand full comment

judaism and mormonism would seem to contradict this, with their most memetically potent form taking a maximalist approach to identity markers, right?

Expand full comment

You are confusing intensity/maximalism with wideness. Some of these intense identity groups are narrow in what they allow to become part of their identities, which allows them to adjust better to changes in geography, politics, business practices, etc

Expand full comment

Even more so, the Amish. Also the military, academics, and the rationalist community.

Expand full comment

The difference between who you are and what you control is simple: "who you are" is your set of core preferences, and what you control is whatever can help you fulfill those preferences. You are unwilling to change "who you are" because you are only willing to do things that help fulfill your preferences, and changing your core preferences does not help fulfill your preferences.

Which of your preferences are your core ones, though? Most of our preferences are instrumental; we hold preference A only because holding preference A helps us fulfill preference B. So we would be willing to change preference A, provided we became convinced that doing so would be better at fulfilling preference B. For instance, preference A could be "make our customers happy," and preference B could be "make a profit." A core preference is a preference that we could never be convinced to change, no matter how rational and knowledgeable we might become.

I think our real core preference is something like inner harmony of our own minds. Everything else is instrumental to achieving mental harmony. And that, therefore, is who we are, ultimately: minds that want harmony.

Expand full comment

Preferences can be wide, with details encompassing many topics, or narrow simple and general, like "survivie".

Expand full comment

"Survive" as a preference is only simple when interpreted as individual survival. And even then, survival of "the self" is just as complicated as "the self" is, which is what we're talking about; "who you are" can be complicated.

If you mean survival of "the species" or "your descendants," and I think you do based on your other writings, then it gets even more complicated because of the question of which future entities count as "your descendants" and how they should be weighted. Do you prefer three descendants with IQ 150 who are honest, or ten descendants with IQ 90 who are nasty criminals, or a million descendants with the intelligence of rats?

Expand full comment

Like for instance, say that you have the option to create a ruthless, inhuman AGI that will rapidly replicate and wipe out the human race. If your only preference is survival of your descendants, would the AGI be considered your descendants, so you would prefer to create it?

If not, then your supposedly "simple" preference for survival of your descendants really bundles a lot of traits together about what it means to be human, and is not simple at all.

Expand full comment

A problem, I think, is that the distinction between a core and instrumental preference is not necessarily stable over time. Through evaluative conditioning a successful means can take on the hedonic properties of the end reward toward which it is aimed. This can be measured directly in the firing rates of midbrain dopamine neurons. When a reward is reliably predicted by a once-neutral cue, the cue comes to elicit the dopamine response that used to be elicited by unexpected presentations of the reward, while the reward, now predictable, elicits no response. The cue (or behavior, in an operant paradigm) has stolen the reward’s significance and become an end/core preference in itself. I suspect this is the rule, rather than the exception, re: how things become parts of our identities in the first place.

Expand full comment

That's a real process, but don't you think a rational person would be capable of recognizing that it happened to them, and deciding the original preference was the core one, not the conditioned preference? "Undoing" the self-delusion. People do lose interest in hobbies, for instance, when the hobbies no longer bring them the joy they once did. So the hobby - a conditioned preference - was not really a core preference.

Expand full comment

I don’t know, tbh, how susceptible the process is to top-down control. I suspect it may take some clever engineering of one’s local environment to keep a well-predicted reward engaging. Of course, people will *say* the original preference is still their core preference and insist all the attention and motivation they’re directing at the “instrumental” preference is purely strategic, but the proof will be in what they do when they can satisfy one preference only by contravening the other, and I’ve seen an awful lot of “rational” and otherwise smart people fail this test.

Expand full comment

> I’ve seen an awful lot of “rational” and otherwise smart people fail this test.

The way you say that though, makes it clear you don't think they're really being rational there. A core preference is what you would want if you were fully rational and informed about everything.

Expand full comment

"What you control is whatever can help you fulfill those preferences"

That statement doesn't make sense. If people controlled whatever could help them fulfill their preferences, then everyone's preferences would be fulfilled, right?

Expand full comment

That's not what I meant. I was speaking in the context of what distinguishes what you control from who you are, not in the context of what distinguishes what you can control from what's outside your control.

A more fully accurate statement would be, *under the assumption that X is either part of your identity or something you control,* X is part of your identity if it is part of your core preferences that all your other preferences merely serve. X is just something you control, and not your identity, if changing X (or leaving it the way it is) is something you do to help fulfill your core preferences.

Expand full comment

Re. "Some hope that a world government will end prevent strong competition in the future, and thus allow the persistence of wider identities." --

I think the people who hope for a world government expect everyone to have mostly identical identities. There will be no more religions (except as cultural decorations), no more cultural beliefs (culture will be purely decorative, e.g., food, clothing, musical styles), no more nominalists, no more empiricists, no more liberals, no more conservatives, no more individualists or free-thinkers, and so one. The difficulties you attributed to having a "wide identity" aren't difficulties if everyone believes and wants the same things.

Expand full comment

This. The world government will be happy to let you have any identity you want, to paraphrase Ford, as long as it’s approved.

Expand full comment

Some memorable, even axiomatic, quotes in this post. High value.

Expand full comment

Hard to disagree, though I think most identity is supposed to carry a unique esthetic to it. So what is perhaps rational is not good advertising.

Expand full comment

take one simple idea and take it very seriously

Expand full comment