Discussion about this post

User's avatar
UWIR's avatar

You are suggesting that checks and balances, death threats, & police can't work for ems as they work for us.1. I'm saying that their effectiveness with regard to humans has limited evidentiary value towards the question of whether they will work with regard towards ems. You said "As ems are basically human minds in robot brains and bodies, any control argument for rejecting them would also seem to be an argument for rejecting humans." I find you to have the burden of proof in this matter. It is not for me to prove that ems definitely will kill us all, it is for you to prove they won't. It is not enough to make an argument from ignorance of "We can't see any clear different between ems and humans, so we should just assume that arguments regarding one apply equally to the other." 2. I disagree that I didn't give any reason why the checks and balances would not apply the same. I gave the lack of equivalence of imprisonment and death penalty as examples. 3. I don't think much explanation is needed for the claim that humans would not be able to police ems the same as humans. How would humans punish ems, if ems have advantages over humans? You'd want the em you punish to be the "same" as the one that committed the offense, but what does it mean to be "the same"? Do you punish just the instance that committed the offense, or all instances of "the same" em? On top of this, just because one population is drawn for another population doesn't mean that statements about one apply equally to the other. All sociopaths are humans, but that doesn't mean that any arguments against giving power to sociopaths apply equally well to giving power to humans in general.

A neg, weight on harming others is the same as a big pos. weight on helping them.No, it's not. One can have a utility function that treats deviations from the status quo asymmetrically. Yes, it would require there being some concept of "status quo", but given that humans can understand the concept, it follows that a human-level AI can as well.

PS: Is the set of tag generators at the bottom of the comment box determined by Disqus or the site owner? I'm not seeing the blockquote option.

PPS: You use the word enormity apparently referring to the task of working towards AI safety. This word, at least from a prescriptivist point of view, refers to a large *bad* thing.

Expand full comment
RobinHanson's avatar

You are suggesting that checks and balances, death threats, & police can't work for ems as they work for us. But you don't say why.

A neg, weight on harming others is the same as a big pos. weight on helping them.

Expand full comment
30 more comments...

No posts