We know truth-seekers should not knowingly disagree about facts. Many are eager to justify their disagreements, however, by noting that values infuse most common disputed topics, such as politics, morality, music, and so on. And yes, there may be nothing irrational about you preferring chocolate while another prefers strawberry. But while it may not be problematic to have differing
I don't know if I'm having a dumb day, or if this post simply uses words with definitions that are very different than what I assume.
In any case, I cannot even work out the structure of the argument given here, never mind whether I agree with it or not. As far as I can tell, its gibberish.
This has happened in previous posts and I've usually been able to track it down to a set of underlaying assumptions that I simply don't have. That may be the case here as well, but if so its on a far grander scale than any case I've seen before.
The semantics of values is intrinsically fact-dependent. If you are uploaded in the virtual world, where each time you eat an apple for breakfast, a child is automatically killed in the real world, and you don't know it, you will make decisions that will lead to killed children. Does it change your values? In what sense it doesn't?
"So is there anyone who thinks themselves more rational to argue about values than to argue about facts?"I'm confused by this question. I thought you clarified your agreement with some comments above that arguments about values are arguments about the underlying facts or context on which those values are based.
In other words, arguments about values are arguments about facts. If so, how can they be more or less rational than arguments about facts...which they are anyway.
Eliezer, yes let's accept impossible worlds as states. It is arguing about "facts" in the sense I defined in the post above, but it may also be "arguing about values" as well.
If you permit "state" to refer to impossible possible worlds, then I think we're essentially in agreement about how to formalize this to the extent it can be formalized. But in what sense is this not "arguing about values"?
Eliezer, if you could have in principle anticipated that your "terminal" values and behavior would have changed in response to such arguments (and changed to something else in anticipation to other arguments), even if you did not actually bother to so anticipate, then your values seem clearly state-dependent values, where info is relevant to figuring out the values. This fits fine in the standard framework. If you could not anticipate such changes even in principle, but all your other behavior is as if you could, well I'd say you are as above except for a defect in your anticipation abilities.
Robin, humans not only seem to commonly argue about terminal values (things done for themselves and not their consequences), but also seem to occasionally update their professed beliefs about terminal values as the result of such arguments, and even, every now and then, change their actual behavior. Is this a malfunction of rationality? What is the truth that destroys it?
PK- I think we can safely describe 'values' as 'general societal preferences,' though I might be wrong. The problem with preferences arises when they involve the actions of *other* people, not just yourself. It's fine to abandon the idea of values if you want to sit under a tree and meditate all day like Siddartha, but if you think that *other* people should be good to each other, then you will need to inflict your values upon them. You can tell them the facts, "Tying that child to a flagpole and dumping bleach down his pants was a very hurtful thing, which given our latest research seems like it might have damaged him for life, and generally nasty little boys like you are doing it in order to assert dominance, and truth be told, it seems to work- but society as a whole will be nicer if everyone stopped doing hurtful things like torturing the other boys in the school yard........."
Lets see how far this argument will get you.
Or, "What you did was evil and you are a very bad person for having done it, and Jesus will hate you and you will go to hell and burn in hellfire for all eternity if you don't repent and promise never to do that again. Now go to your room- you are grounded for a week, daddy will be up there shortly with the switch."
Not that I'm advocating that *we* should use the latter style of impressing values upon people. I don't think it's much different than what the Party of the Right was doing. But I am making a point that facts are not always able to resolve disputes/change behaviors...
Unless you are suggesting that the facts will add up to an objective morality... Plato's "the Good"???
Values can be dissolved into facts. eg. "Chocolate is the best." Can become "I like chocolate the most." or "Most people prefer chocolate." or whichever objective thing the value actually represents. If you don't know what the value represents, ask why your brain produces the value in the first place. After this you only have to deal with facts.
Values are not for arguing about- they are for convincing other people that they'll be happier if they share your own--whether or not you actually think they will be. A good friend of mine became the president of the Party of the Right at Yale. She was a very philosophical-abstract sort of person, though I generally disagreed with most of what she said. As irony would have it, she was an atheist, but still touted Christian fundamentalist doctrine whenever she could as a political strategy. She justified this as a morally valid position under the grounds, "Whether or not Christianity is correct, wouldn't the world be a much better place if everyone believed in it, at least in the form I so aggressively defend?" Whether or not anyone wants to take a stab at answering her question, I thought her hypothetical unrealistically absurd to try to even try to accomplish, and thus her argument invalid. If she wanted to say 'if MORE people believed it...' that would be another question, one I would still disagree with, being the good little truth-seeker I am...
So yeah- inflicting one's values on others is generally a selfish activity, though we don't have many other options if we want to write our own commandments...
So is there anyone who thinks themselves more rational to argue about values than to argue about facts? I thought some might disagree with my main claim here.
Eliezer:hmm, If I concede that, I may as well retract my previous 'No'. But I'm going to say no again:
It only has to believe that its values might in principle be discoverable. It does not need to have a process.It doesn't even have to know how to add.
In order to have that value-finding-process making process, the agent would have to construct it. If the process were built into the agent 'at birth', we would abstract it into its primary goal system, if it ever returned.
In order to construct such a process, or to successfully run it, the agent first has to have a goal of figuring out the right answer.
Consider the value-finding-process making process (or the process that makes it). If it searches a finite region of the possible process space (such as all process our agent could execute) and returns null or an error, then we are left where we started. If it searches the entire process space, then our agent will probably die before it returns. And this assumes that it has the cognitive ability to simply run such a process [without error].
In order to serve its possible-true-value the agent has to have the goal of finding its possible true value even if that takes more time, energy, ram, or basic knowledge than it currently has available to it. Even if it thinks it has a magic value finding process, it might need to eat while it runs.
If the agent is a colossally stupid product of evolution it might take millennia of collaboration to even get to the concept of a value-finding-process making process.
I can't believe nobody injected Thomas Kuhn into this discussion yet. Any set of values gives a higher weight to certain facts over others - we feel a fact matters, or doesn't. Common argument pattern: yuo state a fact that matters to you, opponent responds with "yes, but..." followed by an unrelated fact that's important to them. Political example: lefties think Hitler is the ultimate indictment of the Right, righties think Cambodia is the ultimate indictment of the Left, and neither side can ever convince the other.
James: Then the agent must already know some computation such that if it returns an output, that output is a computation which, if it returns an output, that output is a value.
But they are still ideally responsive to reasons -- that it, it's the case that an agent who is exercising the optimal cognitive operations will modify his/her values in response to appropriate reasons.That should have observable consequences, making the question of what "true values" are one of empirical fact.
Let's put it this way: any concept that has absolutely no basis in empirical fact at some level is contentless and has no meaning.
I don't know if I'm having a dumb day, or if this post simply uses words with definitions that are very different than what I assume.
In any case, I cannot even work out the structure of the argument given here, never mind whether I agree with it or not. As far as I can tell, its gibberish.
This has happened in previous posts and I've usually been able to track it down to a set of underlaying assumptions that I simply don't have. That may be the case here as well, but if so its on a far grander scale than any case I've seen before.
The semantics of values is intrinsically fact-dependent. If you are uploaded in the virtual world, where each time you eat an apple for breakfast, a child is automatically killed in the real world, and you don't know it, you will make decisions that will lead to killed children. Does it change your values? In what sense it doesn't?
"So is there anyone who thinks themselves more rational to argue about values than to argue about facts?"I'm confused by this question. I thought you clarified your agreement with some comments above that arguments about values are arguments about the underlying facts or context on which those values are based.
In other words, arguments about values are arguments about facts. If so, how can they be more or less rational than arguments about facts...which they are anyway.
Eliezer, yes let's accept impossible worlds as states. It is arguing about "facts" in the sense I defined in the post above, but it may also be "arguing about values" as well.
If you permit "state" to refer to impossible possible worlds, then I think we're essentially in agreement about how to formalize this to the extent it can be formalized. But in what sense is this not "arguing about values"?
Eliezer, if you could have in principle anticipated that your "terminal" values and behavior would have changed in response to such arguments (and changed to something else in anticipation to other arguments), even if you did not actually bother to so anticipate, then your values seem clearly state-dependent values, where info is relevant to figuring out the values. This fits fine in the standard framework. If you could not anticipate such changes even in principle, but all your other behavior is as if you could, well I'd say you are as above except for a defect in your anticipation abilities.
Robin, humans not only seem to commonly argue about terminal values (things done for themselves and not their consequences), but also seem to occasionally update their professed beliefs about terminal values as the result of such arguments, and even, every now and then, change their actual behavior. Is this a malfunction of rationality? What is the truth that destroys it?
Have you read Schneier's take on this or the article he sites?
http://www.schneier.com/blo...
PK- I think we can safely describe 'values' as 'general societal preferences,' though I might be wrong. The problem with preferences arises when they involve the actions of *other* people, not just yourself. It's fine to abandon the idea of values if you want to sit under a tree and meditate all day like Siddartha, but if you think that *other* people should be good to each other, then you will need to inflict your values upon them. You can tell them the facts, "Tying that child to a flagpole and dumping bleach down his pants was a very hurtful thing, which given our latest research seems like it might have damaged him for life, and generally nasty little boys like you are doing it in order to assert dominance, and truth be told, it seems to work- but society as a whole will be nicer if everyone stopped doing hurtful things like torturing the other boys in the school yard........."
Lets see how far this argument will get you.
Or, "What you did was evil and you are a very bad person for having done it, and Jesus will hate you and you will go to hell and burn in hellfire for all eternity if you don't repent and promise never to do that again. Now go to your room- you are grounded for a week, daddy will be up there shortly with the switch."
Not that I'm advocating that *we* should use the latter style of impressing values upon people. I don't think it's much different than what the Party of the Right was doing. But I am making a point that facts are not always able to resolve disputes/change behaviors...
Unless you are suggesting that the facts will add up to an objective morality... Plato's "the Good"???
Values can be dissolved into facts. eg. "Chocolate is the best." Can become "I like chocolate the most." or "Most people prefer chocolate." or whichever objective thing the value actually represents. If you don't know what the value represents, ask why your brain produces the value in the first place. After this you only have to deal with facts.
Values are not for arguing about- they are for convincing other people that they'll be happier if they share your own--whether or not you actually think they will be. A good friend of mine became the president of the Party of the Right at Yale. She was a very philosophical-abstract sort of person, though I generally disagreed with most of what she said. As irony would have it, she was an atheist, but still touted Christian fundamentalist doctrine whenever she could as a political strategy. She justified this as a morally valid position under the grounds, "Whether or not Christianity is correct, wouldn't the world be a much better place if everyone believed in it, at least in the form I so aggressively defend?" Whether or not anyone wants to take a stab at answering her question, I thought her hypothetical unrealistically absurd to try to even try to accomplish, and thus her argument invalid. If she wanted to say 'if MORE people believed it...' that would be another question, one I would still disagree with, being the good little truth-seeker I am...
So yeah- inflicting one's values on others is generally a selfish activity, though we don't have many other options if we want to write our own commandments...
So is there anyone who thinks themselves more rational to argue about values than to argue about facts? I thought some might disagree with my main claim here.
Eliezer:hmm, If I concede that, I may as well retract my previous 'No'. But I'm going to say no again:
It only has to believe that its values might in principle be discoverable. It does not need to have a process.It doesn't even have to know how to add.
In order to have that value-finding-process making process, the agent would have to construct it. If the process were built into the agent 'at birth', we would abstract it into its primary goal system, if it ever returned.
In order to construct such a process, or to successfully run it, the agent first has to have a goal of figuring out the right answer.
Consider the value-finding-process making process (or the process that makes it). If it searches a finite region of the possible process space (such as all process our agent could execute) and returns null or an error, then we are left where we started. If it searches the entire process space, then our agent will probably die before it returns. And this assumes that it has the cognitive ability to simply run such a process [without error].
In order to serve its possible-true-value the agent has to have the goal of finding its possible true value even if that takes more time, energy, ram, or basic knowledge than it currently has available to it. Even if it thinks it has a magic value finding process, it might need to eat while it runs.
If the agent is a colossally stupid product of evolution it might take millennia of collaboration to even get to the concept of a value-finding-process making process.
I can't believe nobody injected Thomas Kuhn into this discussion yet. Any set of values gives a higher weight to certain facts over others - we feel a fact matters, or doesn't. Common argument pattern: yuo state a fact that matters to you, opponent responds with "yes, but..." followed by an unrelated fact that's important to them. Political example: lefties think Hitler is the ultimate indictment of the Right, righties think Cambodia is the ultimate indictment of the Left, and neither side can ever convince the other.
James: Then the agent must already know some computation such that if it returns an output, that output is a computation which, if it returns an output, that output is a value.
But they are still ideally responsive to reasons -- that it, it's the case that an agent who is exercising the optimal cognitive operations will modify his/her values in response to appropriate reasons.That should have observable consequences, making the question of what "true values" are one of empirical fact.
Let's put it this way: any concept that has absolutely no basis in empirical fact at some level is contentless and has no meaning.