Imagine you are walking in the wild and come across what looks like a big rock painted with the following words: I may look like an ordinary painted rock, but I’m actually a conscious mind. I know about you and your life, and I want to tell you: from the point of view of your values, you should commit suicide. You may have good arguments against your suicide, and I will listen to any arguments you offer, but I know of very good arguments in favor of suicide. I will listen, but if these words have not changed, then my conclusion has not changed.
An interactive rock creates a very high probability for a projector hiding in the bushes. An overly elaborate practical joke is the most likely explanation unless it totally breaks physics.
I would accept the reverse-case as a self-esteem boost, but I wouldn't believe it to be more true.
Any text that is not clearly directly relevant to me *alone* (i.e. the text clearly does not apply to other people) implies it's just a painted rock. If it refers to me alone, it implies it's a painted rock that's been designed to single me out.
In the event of the legend of the smart rock, I'd give it a very small but non-zero belief; so long as other people see different text: "I would not say this to most people." is falsifiable.
Words changing in front of my eyes is clear evidence of 'magic' - or at least, insufficiently understood technology. I would attempt to question the rock; attempt to understand what it believes my beliefs are and what evidence it has for its statement. I would generally believe the rock as much as I would believe anyone who has shown some evidence of actively trying to harm me with allegedly good intentions.
Beating me at chess or trivia? That's no stronger than the changing words.
This is such an absurd scenario, and as such I cannot imagine a situation in which I would even begin to debate the topic. The premise itself is similar to comparing the relative merits of baseball games, and fruit salad.
I understand that I am *supposed* to suspend disbelief; however what a rock and the mammal in this scenario have in common in short summation include: similar molecular composition, similar fixation in time/space, identical observance of physical laws.
A rock's existential experience would be so remarkably different that no "values" can be equated and so no value judgments are valid. To imagine that a rock can have relate-able experiences would be to fundamentally change the nature of a rock. This isn't just a bias, it is a completely nonviable interaction.
Cobb, a practical person would only ever give you good news, as it is not clear you take bad news very well. Yup, nothing but good stuff for you here ...
I take it as axiomatic that humans best understand humans because humans experience life as humans do. Rocks do not. No matter how you might embed a rock with consciousness, it cannot be human consciousness - only rock consciousness which must be held inferior.
I would thus see in the rock some alien intelligence whose purpose is to incite people to self-destruction without a human capacity to empathize. It is therefore a threat to humanity. As such, I destroy the rock.
My values are such that from the point of view of my values, I should never commit suicide. Therefore, I know the rock is "lying" when it says, "I know about you and your life."
Rciii, imagine I told you that I would meet you at a particular address at a particular time. Given all the possible times and places I could have said, the prior probability you would have assigned to that exact combination is very low. Nevertheless, you would probably believe me anyway. So it is not clear to me what an "extraordinary" claim is.
After thinking about this, I'm going to take the rock's side.
There are probably claims that a rock could make that I might accept. Consider the weather rock - "If I'm wet it's raining, if I'm white it's snowing..."
And I'm willing to posit the possibility that there is some sequence the rock could follow to gain my confidence.
After all, saying "I cannot imagine what a rock could do to convince me of something" or "There is no way anyone or anything can convince me of this proposition" is in some part a statement about my imagination and sophistication.
That said, extraordinary claims require extraordinary evidence, and here we have two extraordinary claims. Do evidentiary requirements add or multiply?
The prior that an interactive Suicide Rock is a result of insanity may be so strong as to create an unrebuttable presumption, a closed intellectual system akin to solipsism. Good performance from the rock on the above factors could bring belief up to a maximum, but no further.
If I were to encounter a burning bush that demonstrated godlike abilities, I would have to conclude that my reality is not what it appeared to be, but is perhaps a Matrix-style simulation or a 'basement-level' universe with magic. My priors would favour the former, and no sensory evidence could defeat that judgment, including claims by the bush.
Eliezer, some kinds of advice tend to be given by people trying to exploit you, so it makes perfect sense to be less trusting of someone conditional on hearing that sort of advice. It is not obvious what the rock has to gain from your suicide though.
Rcrii, your feeling uncertain about the rock's priors has nothing to do with the priors being uncommon. In a formal model, priors are always common knowledge; everyone knows that everyone knows ... them.
Maybe part of the problem here is the utter lack of common priors, about anything. This, I think, is where Michaels point come in. I sometimes accept the word of others (doctors, journalists, certain pundits) without delving into their argument because I have knowledge about their reliability from other sources. This rock, on the other hand, contradicts everything I know about rocks, so will have to work extra hard to convince me of anything.
Eliezer's suggestion reminds me of a joke from a Terry Pratchett book:
In the old country there was a magic glade with a tree stump in the middle. If your pony throws a shoe, you can take it to the glade and tie it to the stump and leave a gold piece on the stump. When you come back the next morning the gold piece will be gone ... and so will the pony.
If a dialogue can be established, such I say or write or communicate anything and the rock reacts intelligently, topically, then the rock's position and reasoning could be understood. Still nothing the rock says could lead me to suicide, since I live not because of rational arguments (where the rock may have the upper hand) but because I want to or I dont want the contrary or just lazyness or curiosity or whatever.
following Eliezer's argument, what if the rock - that looks like a lion's head - said "put 1000 $ cash in my mouth and I will make you lucky etc. Would you invest 1 $ to see if it works?
What if the rock doesnt look like a rock but like an ordinary human being and asks for 1000 $?
As we say in Yiddish, if my grandmother had beitzes (testicles) she wouldnt be my grandmother, he would be my grandfather. And what if my grandmother had wheels instead of legs? she would be a byclicle.
Robin, I think James D. Miller has a point. The extremity of this advice obscures many potential reasons for agreement because those reasons are not quite strong enough to justify suicide. What if the rock told you to bury $1000 underneath it at midnight?
James, the fact that there is a large option value to waiting makes it a apriori unlikely that not waiting is the best decision. But we often listen to and believe people who tell us things we initially thought were unlikely.
Michael, if you never believe anyone who has not proven their claim to you, you will have little use for the news media or journals or education or any professional advisor like a doctor.
An interactive rock creates a very high probability for a projector hiding in the bushes. An overly elaborate practical joke is the most likely explanation unless it totally breaks physics.
An early precursor of Suicide Rock.
I would accept the reverse-case as a self-esteem boost, but I wouldn't believe it to be more true.
Any text that is not clearly directly relevant to me *alone* (i.e. the text clearly does not apply to other people) implies it's just a painted rock. If it refers to me alone, it implies it's a painted rock that's been designed to single me out.
In the event of the legend of the smart rock, I'd give it a very small but non-zero belief; so long as other people see different text: "I would not say this to most people." is falsifiable.
Words changing in front of my eyes is clear evidence of 'magic' - or at least, insufficiently understood technology. I would attempt to question the rock; attempt to understand what it believes my beliefs are and what evidence it has for its statement. I would generally believe the rock as much as I would believe anyone who has shown some evidence of actively trying to harm me with allegedly good intentions.
Beating me at chess or trivia? That's no stronger than the changing words.
This is such an absurd scenario, and as such I cannot imagine a situation in which I would even begin to debate the topic. The premise itself is similar to comparing the relative merits of baseball games, and fruit salad.
I understand that I am *supposed* to suspend disbelief; however what a rock and the mammal in this scenario have in common in short summation include: similar molecular composition, similar fixation in time/space, identical observance of physical laws.
A rock's existential experience would be so remarkably different that no "values" can be equated and so no value judgments are valid. To imagine that a rock can have relate-able experiences would be to fundamentally change the nature of a rock. This isn't just a bias, it is a completely nonviable interaction.
Cobb, a practical person would only ever give you good news, as it is not clear you take bad news very well. Yup, nothing but good stuff for you here ...
I go one step further. I destroy the rock.
I take it as axiomatic that humans best understand humans because humans experience life as humans do. Rocks do not. No matter how you might embed a rock with consciousness, it cannot be human consciousness - only rock consciousness which must be held inferior.
I would thus see in the rock some alien intelligence whose purpose is to incite people to self-destruction without a human capacity to empathize. It is therefore a threat to humanity. As such, I destroy the rock.
Or I move it to the territory of my enemy.
My values are such that from the point of view of my values, I should never commit suicide. Therefore, I know the rock is "lying" when it says, "I know about you and your life."
Problem solved!
I consider the claim that I should commit suicide extraordinary. I also consider the claim that a rock has knowledge to impart extraordinary.
Are you suggesting that the claim that i should commit suicide is reasonable on the basis that there are only two choices (to be or not to be)?
Rciii, imagine I told you that I would meet you at a particular address at a particular time. Given all the possible times and places I could have said, the prior probability you would have assigned to that exact combination is very low. Nevertheless, you would probably believe me anyway. So it is not clear to me what an "extraordinary" claim is.
After thinking about this, I'm going to take the rock's side.
There are probably claims that a rock could make that I might accept. Consider the weather rock - "If I'm wet it's raining, if I'm white it's snowing..."
And I'm willing to posit the possibility that there is some sequence the rock could follow to gain my confidence.
After all, saying "I cannot imagine what a rock could do to convince me of something" or "There is no way anyone or anything can convince me of this proposition" is in some part a statement about my imagination and sophistication.
That said, extraordinary claims require extraordinary evidence, and here we have two extraordinary claims. Do evidentiary requirements add or multiply?
The prior that an interactive Suicide Rock is a result of insanity may be so strong as to create an unrebuttable presumption, a closed intellectual system akin to solipsism. Good performance from the rock on the above factors could bring belief up to a maximum, but no further.
If I were to encounter a burning bush that demonstrated godlike abilities, I would have to conclude that my reality is not what it appeared to be, but is perhaps a Matrix-style simulation or a 'basement-level' universe with magic. My priors would favour the former, and no sensory evidence could defeat that judgment, including claims by the bush.
Eliezer, some kinds of advice tend to be given by people trying to exploit you, so it makes perfect sense to be less trusting of someone conditional on hearing that sort of advice. It is not obvious what the rock has to gain from your suicide though.
Rcrii, your feeling uncertain about the rock's priors has nothing to do with the priors being uncommon. In a formal model, priors are always common knowledge; everyone knows that everyone knows ... them.
Maybe part of the problem here is the utter lack of common priors, about anything. This, I think, is where Michaels point come in. I sometimes accept the word of others (doctors, journalists, certain pundits) without delving into their argument because I have knowledge about their reliability from other sources. This rock, on the other hand, contradicts everything I know about rocks, so will have to work extra hard to convince me of anything.
Eliezer's suggestion reminds me of a joke from a Terry Pratchett book:
In the old country there was a magic glade with a tree stump in the middle. If your pony throws a shoe, you can take it to the glade and tie it to the stump and leave a gold piece on the stump. When you come back the next morning the gold piece will be gone ... and so will the pony.
If a dialogue can be established, such I say or write or communicate anything and the rock reacts intelligently, topically, then the rock's position and reasoning could be understood. Still nothing the rock says could lead me to suicide, since I live not because of rational arguments (where the rock may have the upper hand) but because I want to or I dont want the contrary or just lazyness or curiosity or whatever.
following Eliezer's argument, what if the rock - that looks like a lion's head - said "put 1000 $ cash in my mouth and I will make you lucky etc. Would you invest 1 $ to see if it works?
What if the rock doesnt look like a rock but like an ordinary human being and asks for 1000 $?
As we say in Yiddish, if my grandmother had beitzes (testicles) she wouldnt be my grandmother, he would be my grandfather. And what if my grandmother had wheels instead of legs? she would be a byclicle.
The Gemoyre says... enough.
Robin, I think James D. Miller has a point. The extremity of this advice obscures many potential reasons for agreement because those reasons are not quite strong enough to justify suicide. What if the rock told you to bury $1000 underneath it at midnight?
James, the fact that there is a large option value to waiting makes it a apriori unlikely that not waiting is the best decision. But we often listen to and believe people who tell us things we initially thought were unlikely.
Michael, if you never believe anyone who has not proven their claim to you, you will have little use for the news media or journals or education or any professional advisor like a doctor.