Is Forgotten Party Death?

Consider 3 possible space-time experience trajectories:

ForgottenPartyToSpurIn A, a person takes a drug at the start of a party which causes them to not remember that party the next day or anytime later. In C, an em splits off a short term “spur” copy which does a short term task and then ends.

Scenario B can be seen as like A, except that right after taking the drug the person quickly moves to a distant party location. Call this B1.  Alternatively, B can be seen as like C, except that the em original is archived and inactive while the spur works. Call this B2.

In my talks on Age of Em, I’ve heard many object to scenario C, but few object to A. In scenario A, few think they’d be stressed near the end of the party, thinking they are about to “die.” Yet many see scenario C as a “death,” and claim the em spur would refuse to do their assigned task, and instead fight fiercely to keep going.

Scenario B is designed to be intermediate between A and B, and so to force a choice between conflicting intuitions. Would you really see B2 as “death” but B1 as no big deal, even though they have the same space time structure of experiences? If not, I think you should admit either that A is “death”, or that C is not. Or explain what matters besides the space-time structure of experience.

I confidently predict that ems see all of these as no big deal, because a competitive em world selects for ems who are more productive, and a willingness to create short term spurs is quite productive in the em world.

Note that this issue engages similar space vs time morality intuitions as these three prior posts.

GD Star Rating
loading...
Tagged as: , ,
Trackback URL:
  • Daniel Carrier

    > I confidently predict that ems see all of these as no big deal, because a competitive em world selects for ems who are more productive, and a willingness to create short term spurs is quite productive in the em world.

    The problem with this is that it’s in no way dependent on what’s actually correct. You could be wrong about how ems would act. You could be wrong about what’s ethical. You could be wrong about both. The first two options are a little frightening, except the second.

    If you want my opinion, there’s nothing fundamentally different about all the experiences making a line. I think it’s important to pick one 100-year person over 100 1-year people if that’s what they want, but if you select for ems that don’t care it doesn’t matter.

  • Dave Lindbergh

    To me, all 3 scenarios involve “death” – any time one has experiences that will not be remembered, the point at which the memory is lost is a “death”.

    If Bill Murray (Groundhog Day) didn’t remember each previous day, he’d “die” at the end of each day.

    But who cares, really, if “many object” to C? Nobody has to do it if they don’t choose to.

    It’s the few who don’t object that will be doing it.

    • Dave Lindbergh

      (Replying to myself…) After a little more thought, “death” can be seen as the same as loss of memory, in general.

      I no longer remember 97%+ of what I did in high school 40 years ago. That “me” is 97%+ dead already.

      There’s a new “me” now.

      So be it. I don’t find it too upsetting. If the rewards were right, I might well volunteer to be spurred.

      • lump1

        The question is, would you volunteer to be spurred, and then killed 5 hours later and have your spur live on. That is how the matter will seem to the spur in the thing you said you’d be OK with. I suspect you’re willing to be a sadist but not a masochist – understandably, until you realize that in either case, the victim will be exactly like you.

      • Dave Lindbergh

        I’m well aware of what the question is.

        As I said, if the rewards were right, I might well volunteer to be spurred.

        If “life” is the same as memory, then the spur loses only a few hours of life – the last few since it was forked off.

        I’m not sure that’s any worse than wasting a day procrastinating. (Which for me happens too often already.)

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        I no longer remember 97%+ of what I did in high school 40 years ago. That “me” is 97%+ dead already.

        Then, an individual with a better episodic (biographical) memory is more alive? (Episodic memory: https://en.wikipedia.org/wiki/Episodic_memory )

        I think your idea is interesting, but the kind of memory involved isn’t primarily episodic memory. I would care little if all my biographical memories of my junior high school years were obliterated, but I’d care a lot if what I learned from those experiences were erased.

  • Lord

    People are used to killing time for pay, called work, and to working for the benefit of others, dependents, but would they work as slaves to their other self. The other self may like this as they benefit without paying the cost but would you pay the cost without any benefit yourself? Only if they saw themselves as one which would become increasingly difficult the longer they are apart, and if they benefit themselves, their first priority would be insuring their continued existence. Nor is there any benefit in losing whatever they have learned in the course of performing their tasks but the longer they are turned off, the less relevant they become. People don’t worry much about what they don’t know will happen and they may not worry much about what happens to others, but the same lack of worry about what happens to their em leads their em to not worry about what happens to the original so no they won’t knowlingly and willingly be their slave while if you care for your em, you will want their continued existence as they would yours. It would be incredibly difficult to see all copies as one whole when disposing of others as having served their purpose.

    • http://overcomingbias.com RobinHanson

      Every work day your evening self “enslaves” your day self, forcing the day self to work so the evening self can enjoy itself.

      • Lord

        Sure, but you don’t consider them distinct. You know you must work for an income so you can survive and enjoy yourself. If you didn’t think you would survive longer than your work and had no chance of ever enjoying yourself, would you still work? Only if the work itself was not work but enjoyable.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        If you didn’t think you would survive longer than your work and had no chance of ever enjoying yourself, would you still work?

        For a period of time, you would identify your interests with your original because you would be subject its thoughts, including its plans.

        The transition to being subjectively a separate entity cannot be instantaneous – if only because nothing is.

        How long is that transition period? The party analogy provides a subjective metric.

      • Lord

        Or it could grow, seeing an immortal em as your partner to work towards common goals and heir to live on after.

      • Lord

        Once you are capable of independent experience, you are no longer the same individual. Show me a twin willing to labor for the sole benefit of their twin with no expectation of benefit. People did live under slavery, but it was always brutal and resented even under the best circumstances. There was always some chance of respite or hope of freedom and people will do much to survive, abiding their time, but desperation can thwart this.

  • Vamair

    While it’s intuitively plausible that the C scenario is okay, I’m not sure “ems would find nothing wrong with that” is a good reason to think so, as it’s possible no evolved em would care much about anything we care about now and even be replaced with programs lacking any sense of self if the sense of self is eating resources without a corresponding productivity boost.

    • http://overcomingbias.com RobinHanson

      But I didn’t say that ems finding C okay is a reason its okay.

  • lump1

    It seems to me that the spur in B would be justified in thinking that they will survive the episode, that the resumed solid line would be “me” (that won’t remember what I’m doing now).

    In the case of C, there is no psychological room to think that will be “me”, because it’s clear that the survivor will be “that guy” – who’s probably working right now in the cubicle next to you. And, as Wittgenstein said, I can’t have a pain in [his] tooth.

    You might respond that the thinking I described in B is self-deceptive, but my point is, human-like thinkers are capable of that kind of self-deception in B-like, but not C-like cases. Maybe introducing ignorance would solve the problem in C-like cases, as in, I’m always given reason to think that the other guy is the spur, never me.

  • lump1

    Related thought experiment: The Star Trek transporter destructively scans your body at the source and builds it (from different atoms) at the destination. Awesome tech; only Luddites would protest!

    But now imagine that the scan doesn’t completely disintegrate your body at the source, but only irradiates it enough so that you are sure to die in 1-6 hours. An instant euthanizer is on by default, but that default can be overridden in special circumstances, like when the chief engineer is briefly needed in two places.

    How would we feel about the transporter now? Differently, I’d wager – yet the difference in the functioning is very small. We can also imagine intermediate cases (euthanasia plus instant acid bath at the source).

  • endril

    The little gap in the A/B1 scenarios represents a mental transformation of a single creature, in the B2/C scenarios it represents there being one less conscious creature.
    Do you think humans are really indifferent between “you’ll wake up with amnesia tomorrow” versus “you won’t wake up tomorrow?”
    You’re probably right that ems will be okay with it, in any case.

    • Robert7

      For what it’s worth, I’m human and indifferent between “You’ll wake up with amnesia tomorrow” and “an identical copy of you with your current memories will wake up tomorrow and you won’t”.

  • Patrick

    I’m not sure if you’re already aware of this, but Greg Egan has written many short stories and novels along these lines (usually portraying what you call “spurs” as being not a big deal to the transhumans creating them). See for instance “The Planck Dive” and “Permutation City” for two somewhat representative examples.

    • http://overcomingbias.com RobinHanson

      I have read many Egan novels.

  • http://juridicalcoherence.blogspot.com/ Stephen Diamond

    How long does a “short term” spur continue? This matters a great deal, which demonstrates the disanalogy to death.

    A spur still controlled by the original intention won’t resent being turned off. A spur that has time to develop its separate and conflicting intentions will. Similarly, I’d sacrifice a night (as in the party example) for small compensation, but forgetting 20 years would differ in its subjective significance.

    • http://overcomingbias.com RobinHanson

      A spur lasts less than ten hours, as do most parties.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        I wonder how members of your audience who forecast problems with C would answer if they were asked to specify the maximum duration for the spur that would be unproblematic. Would they really say zero, as their metaphysics or morals apparently requires?

      • http://overcomingbias.com RobinHanson

        Yes, I think most would really answer zero.

  • AJ Li

    Wait – but in the party scenario, you know your actions during the party also affects your future self —

    Well yes, the same applies in the em scenario. If you’re someone who won’t work in such a scenario, your master em won’t be continued.

    But while I care about my future self, *I* don’t think of the other copies of myself the same way, logic be damned. Because me and the other copies exist simultaneously, it’s impossible for me to make the same emotional connection towards their well-being.

    That doesn’t matter, we’ll select for the people who will have emotional attachment to other copies of their own ems.

    What if there are multiple branches? I know that I’m just one branch among a potentially infinite number of branches all doing some sort of drudgery. Since I care about the well-being of my fellow copies, continuing to exist will cause a potentially infinite amount of suffering in my copies, in exchange for a negligible reward in the master copy. Using Yudkowsky’s dust specks/torture analogy, this is analogous to living a life while being tortured forever. I’d rather self-terminate.

    We’ll select for people who won’t be bothered by this sort of detail, or find drudgery pleasurable.

    It still seems to me that you’re just breeding for a bunch of people who would be happy in slavery, Robin.

    • lump1

      First of all, Robin is making a (conditional) prediction, not advocating something. Even depressing predictions might be right. Second, the ems in his picture are not generally slaves. They work for wages like we do. Yes, they work much harder, because paying “rent” for cpu time is literally a matter of life and death, but they are also far better at their jobs and more into it. Slackers like us can’t make it in the em world. Even with 160 IQ and serious grit, you will be always be outbid for cpu cycles by a copy of the guy with 210 IQ who loves toil so much that he hardly ever has a stray thought. Since copying is cheap, a copy of him will earn more per cpu cycle than you can, so nobody will offer cycles you can afford, so you’re dead. But the trillions who live will be pretty amazing people – though nothing like anyone whom we would want as friends.

      • AJ Li

        I’ve been reading his writings for quite a few years now, but thanks for the summary I guess.

        The point of my post was not to critique the morality. The point I was bringing across is that that a straight comparison of scenarios A B and C above don’t work because branches can be potentially infinite compared to linear time. Therefore, in order for an em to be productive, significant ‘breeding’ will have to take place across successive generations of ems, effectively rendering ems no longer comparable to modern day humans in their decision making, which again makes the the comparison in this post moot.

  • Contrarian

    At least some people take the view that death is bad mostly because people are unable to enact their values after death. An em might get around this by reasoning that a similarly-emulated mind would still share their particular values set (me tomorrow, post-party is still much more similar to me, value-wise, than any other person in the world).

    Another view: one can have memories and have a great deal of ability to think and still not be conscious. Most of our thinking is much more similar to computing that we cannot introspect, whereas “conscious thought” (for lack of a better term) is qualitatively different. I don’t think psychological continuity need be boiled down so simply as continuity of memory, although a permanent and complete memory transplant (as some thought experiments would have it) although not severe amnesia (now that I think about it, there’s a lot to memory that isn’t “memories that we can recall or imagine in our mind’s eye”) likely destroys any meaningful notion of identity (trying not to get too sloppy with metaphysical claims here).

  • hypnosifl

    Given that there has probably been very little growth of new simulated synapses in the time the “main” em and the spur have been separated, differences in their memories of the period since being separated are likely to be primarily just a matter of different synaptic “weights” of some kind (different concentrations of simulated molecules at the synapses, perhaps). So as an alternative to wiping out the spur, why can’t the two versions just re-merge by averaging together the weights at each synapse? If they wanted the memories of the boring task of the spur to be “fuzzier”, less prominent, they could also do a weighted average, so for example the combined concentration of a simulated molecule at a synapse could be something like 0.95*(value for main em) + 0.05*(value for spur). This would allow the spur to feel that at least part of his stream of experience would be re-integrated, even if only at a subconscious level.

    I think this would also be a closer analogy to a realistic memory-erasure drug, which likely doesn’t wholly remove all traces of whatever changes happened to your brain during the period that’s erased from conscious recall. I suspect a lot of people who understand something about how the brain works (and don’t believe in a supernatural soul), and who’d be ok with the idea of taking some real-world memory-erasing drug after a party, would feel more reluctant to go to a party and then undergo a hypothetical memory-erasure procedure that reset their brain to exactly the state it was before the party, which is more clearly like condemning the version of you at the party to “death”.

    • lump1

      Interesting solution! Maybe just before “spur death” the spur would make a brief inventory of what memory content from their minilife they want to preserve, and this could get merged into the singular subjectivity. That could even become codified as a standard practice in starting and stopping spurs.

  • free_agent

    You could set up the same scenarios using a cloning machine — Is it a bad thing if I make a clone of myself, send it to a party, and then kill it when it gets back?

    IMO, the central problem is that we’ve never faced this sort of situation before, so neither our genes nor our memes are selected to deal with it adaptively. Not surprisingly, people say a lot of odd things.

    One thing that makes EMs different from clones is that EMs don’t have genes, so there’s no inter-genomic conflict between genes and memes — genes would have an incentive to build brains that cannot carry memes that are harmful to the genes.

    • AVT

      They don’t have genes only in the trivial sense that red blood cells don’t have genes.

      If they’re truly EMs and not generic AIs they’ll have to incorporate the effects of the chromosomal regulation of nerve cells and thus experience the same conflicts that biological minds have.

  • http://invariant.org/ Peter Gerdes

    Yet another reason (like the many points about abortion) not to believe death is (in and of itself) morally significant (only the fear of death and grief it causes is).

    A good utilitarian shouldn’t care which is labeled death and which isn’t.

    • Hedonic Treader

      A good utilitarian shouldn’t care about people at all. Just do whatever works best, tax the resulting economy to maximize revenue, and use it to fund hedonium.

      Of course, practically no one wants to be a good utilitarian. There’s almost an anti-consensus against it, even among rationalists and effective altruists, not to mention other people.

    • Jared

      So there’s nothing wrong with a loner committing suicide?

  • Chuckwood

    Like most people, I agree that A is not death. Nor do i believe that C is death, but neither am I convinced that an em is a living being. 

    An em is nothing more than an admittedly very complex simulation of a human brain. It is a state machine that can be archived, cloned, modified, et cetera at will. If I shut down an em and start it back up a month later, has it “died”? See A.

    As a human, I don’t think an em will share the same concept of existence. It would probably not consider deletion of its clone at the end of a job and incorporating the clone’s memory into its own a form of “death”. 

    However, If we assume an em would consider C to be death, then to ensure its survival at the expense of the clone, it would likely limit the clone’s knowledge of its impending demise before spinning up the simulation. Ems would either refuse to clone themselves for these temporary jobs or we’d have a lot of sociopathic ems running things. 

    My own feeling is that ems will be controlled and exploited by corporations and/or government entities and used as cheap labor and tools, like digital slaves.

  • arch1

    Thanks Robin for this scenario description. I’m still confused but it and the ensuing discussion is helping me sort through things. It sparked another question-
    For those of you who in Scenario A, near party’s end, would be OK with the imminent disappearance of your party memory so long as your post-party life resumed otherwise unaffected:
    Would you be similarly sanguine if you became convinced that a) an infinite number of exact copies of your Hubble volume really exist, and b) only your particular copy was about to be zapped out of existence as a special “one-off” event? If not, why not? (After all, in this “zapping” scenario, nothing whatsoever is lost to existence, not even one person’s memory of one possibly-lame party)?

  • arch1

    [retry w/ better formatting, ignore previous]
    Thanks Robin for this scenario description. I’m still confused but it plus the ensuing discussion is helping me sort through things. It sparked another question-

    For those of you who in Scenario A, near party’s end, would be OK with the imminent disappearance of your party memory so long as your post-party life resumed otherwise unaffected:

    Would you be similarly sanguine if you became convinced that a) an infinite number of exact copies of your Hubble volume really exist, and b) [this part is a bit magical but I claim in a way that’s irrelevant to the principle at issue] your particular Hubble volume was about to be zapped out of existence as a special “one-off” event? If not, why not? (After all, in this “zapping” scenario, *nothing whatsoever* is lost to existence, not even one person’s memory of one possibly-lame party)?

  • Pingback: Contra Hanson on Party Death | Nintil

  • http://www.gwern.net/ gwern
  • arch1

    OK I’ll run up the flagpole a rationale for my current gut reaction. (Here I’m assuming that the spur in C was terminated without the permission of the em copy).
    C is wrong and A isn’t, because C (unlike A) involves a human-level consciousness being terminated without its permission and without sufficient extraordinary reasons.

    • arch1

      I was hoping this comment would trigger a response because a) I don’t really trust my gut here, b) more importantly, it *might* have been my only legitimate chance to quote my favorite Moody Blues lyric. Oh well:-)

    • http://juridicalcoherence.blogspot.com/ Stephen Diamond

      But C permits the termination, at least at its inception. The termination is part of the deal, consented to by the original, hence by the spur when it first arises.

      So, the ethical question is whether the right is alienable by prior consent, since the spur might change its mind.

      • arch1

        Stephen, if consent carried over, a callous twisted em could have endless fun injecting spur copies into ever-more-creative torture scenarios, all with their consent of course. Our consent theory needs must at least recognize that spur copy and original are distinct beings with different interests.

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        The original’s problem is to negotiate a deal with his future spur. Original must find a deal acceptable to spur, or spur won’t perform. That’s why I say the spur agrees at the inception. Original can test the acceptability of the deal to the spur against his own intuitions because initially those states can be identical. (Not must be, as I my response might have made it seem. Original must have an incentive to correctly predict spur’s behavior.)

        [As I understand the problem, it is whether the spur will ordinarily be willing to do the original’s bidding at spur’s inception, which willingness or unwillingness is knowable to original. When so, it’s consent. When not, maybe it’s a crime by original.]

      • http://juridicalcoherence.blogspot.com/ Stephen Diamond

        Original can test the acceptability of the deal to the spur against his own intuitions because initially those states can be identical.

        This test is satisfied simply by the original, immediately prior to being spurred, forming an intention to perform. This intention is what “carries over.” Successful intention formation isn’t guaranteed for every job.

      • arch1

        Thanks for the clarification Stephen – I had misinterpreted your “hence”. But as I get into this another concern is raised: To what extent the law should facilitate or even countenance self-termination commitments in the first place.

        My gut feel is that in our current human world the law should at least do nothing to *facilitate* such commitments, and that the advent of ems changes nothing in this regard. But I am interested in your and others’ thoughts (also in how current pre-em law handles this).

  • http://lukeparrish.rationalsites.com Luke Parrish

    There’s a rationalist fanfic of Animorphs that explores this concept, and how different people react differently to being temporarily emulated minds. Highly recommend it. https://www.fanfiction.net/s/11090259/1/r-Animorphs-The-Reckoning

    • Hedonic Treader

      There’s also the em survival horror game SOMA that deals with some of these emotional issues.

  • ohlookitsasquirrel

    As someone with rather severe ADHD, I think I may have something to contribute to the conversation of continuity of experience. I didn’t get medication until later in life, so I spent a few decades simply dealing with my attention span by thinking of myself as more of an operating system — a set of policies, best practices, and long-term goals — than a single agent with a continuous stream of experiences.

    Since I can’t be on meds 24/7 (re: amphetamines are stressful on the cardiovascular system, and I gotta sleep from time to time), I have about 5 conscious hours per day when I have to operate without medication. I consider C to be akin to a case of “losing my train of thought” or –perhaps a more relatable for those with longer attention spans– waking up from a dream. When I wake up in the morning, the “dream me” doesn’t fight to keep sleeping so dream-me can finish the story. If anything, awake-me might choose to finish the plot of my dream by snoozing for a few minutes, but dream-me has no say in the matter, and is entirely impartial to awake-me’s decision.

    In the case of losing my train of thought, I do become frustrated when I can detect that I have lost progress on a task and repeatedly have to start it over and over again until I can make it to the end. That is due to the fact that I feel frustrated by a computational limitation, but as for the issue of “wishing to avoid termination”, I simply accept the situation and try to find ways to remain as operational as possible. But when I start a new task, I really may as well have terminated the version of me that was working on the task before, since all I have to go on is my records and my medium-term memories — not any sense of continuity of experience.