So if said planet decided that its preference was to perish, rather than sacrifice children, would this be irrational?
You can’t decide your preference, preference is not what you actually do, it is what you should do, and it’s encoded in your decision-making capabilities in a nontrivial way, so that you aren’t necessarily capable of seeing what it is.
Compare preference to a solution to an equation: you can see the equation, you can take it apart on the constituent terms, but its solution is nowhere to be found explicitly. Yet this solution is (say) uniquely defined by the equation, and approximate methods for solving the equation (analogized to the actual decisions) tend to give their results in the general ballpark of the exact solution.
You can’t decide your preference, preference is not what you actually do, it is what you should do, and it’s encoded in your decision-making capabilities in a nontrivial way, so that you aren’t necessarily capable of seeing what it is.
The analogy in the next paragraph was meant to clarify. Do you see the analogy?
A person in this analogy is an equations together with an algorithm for approximately solving that equation. Decisions that the person makes are the approximate solutions, while preference is the exact solution hidden in the equation that the person can’t solve exactly. The decision algorithm tries to make decisions as close to the exact solution as it can. The exact solution is what the person should do, while the output of the approximate algorithm is what the person actually does.
I suppose I’m questioning the validity of the analogy: equations are by nature descriptive, while what one ought to do is prescriptive. Are you familiar with the Is-Ought problem?
jwdink, I don’t think Vladimir Nesov is making an Is-Ought error. Think of this: You have values (preferences, desired ends, emotional “impulses” or whatever) which are a physical part of your nature. Everything you decide to do, you do because you Want to. If you refuse to acknowledge any criteria for behavior as valuable to you, you’re saying that what feels valuable to you isn’t valuable to you. This is a contradiction!
An Is-Ought problem arises when you attempt to derive a Then without an If. Here, the If is given: If you value what you value, then you should do what is right in accordance with your values.
But there seemed to be some suggestion that an avoidance of sacrificing the children, even to the risk of everyone’s lives was a “less rational” value. If it’s a value, it’s a value… how do you call certain values invalid, or not “real” preferences?
I missed where Vladimir made that suggestion, though I’m sure others have. You can have an irrational value, if it’s really a means and not an end (which is another value), but you don’t recognize that, and call the means a value itself. Means to an end can of course be evaluated as rational. If anyone made the suggestion you mention, they probably presumed a single “basic” value of preserving lives, and considered the method of deciding to be a means, but denoted as a value.
(Of course, a value can be both a means and an end, which presents fun new complications...)
I agree generally that this is what an irrational value would mean. However, the presiding implicit assumption was that the utilitarian ends were the correct, and therefore the presiding explicit assumption (or at least, I thought it was presiding… now I can’t seem to get anyone to defend it, so maybe not) was that therefore the most efficient means to these particular ends were the most rational.
Maybe I was misunderstanding the presiding assumption, though. It was just stuff like this:
Lesswrongers will be encouraged to learn that the Torchwood characters were rationalists to a man and woman—there was little hesitation in agreeing to the 456′s demands.
Or this, in response to a call to “dignity”:
How many lives is your dignity worth? Would you be willing to actually kill people for your dignity, or are you only willing to make that transaction if someone else is holding the knife?
Haha, we must have very different criteria for “confusing.” I found that post very clear, and I’ve struggled quite a bit with most of your posts. No offense meant, of course: I’m just not very versed in the LW vernacular.
My comments can be confusing, or difficult to get over the wider inferential gaps. In this case I meant that nickernst’s comment could just be expressed much more clearly.
That’s not a particularly helpful or elucidating response. Can you flesh out your position? It’s impossible to tell what it is based on the paltry statements you’ve provided. Are you asserting that the “equation” or “hidden preference” is the same for all humans, or ought to be the same, and therefore is something objective/rational?
Preference of a given human is defined by their brain, and can be somewhat different from person to person, but not too much. There is nothing “objective” about this preference, but for each person there is one true preference that is their own, and same could be said for humanity as a whole, with the whole planet defining its preference, instead of just one brain. The focus on the brain isn’t very accurate though, since environment plays its part as well.
I can’t do justice to the centuries-old problem with a few words, but the idea is more or less this. Whatever the concept of “preference” means, when the human philosophers talk about it, their words are caused by something in the world: “preference” must be either a mechanism in their brain, a name of their confusion, or something else. It’s not epiphenomenal. Searching for the “ought” in the world outside human minds is more or less a guaranteed failure, especially if the answer is expected to be found explicitly, as an exemplar of perfection rather than evidence about what perfection is, to be interpreted in nontrivial way. The history of failure to find an answer while looking in the wrong place doesn’t prove that the answer is nowhere to be found, that there is now positive knowledge about the absence of the answer is the world.
Okay, so I’ll ask again: why couldn’t the humans real preference be to not sacrifice the children? Remember, you said:
You can’t decide your preference, preference is not what you actually do, it is what you should do
You haven’t really elucidated this. You’re either pulling an ought out of nowhere, or you’re saying “preference is what you should do if you want to win”. In the latter case, you still haven’t explained why giving up the children is winning, and not doing so is not winning.
And the link you gave doesn’t help at all, since, if we’re going to be looking at moral impulses common to all cultures and humans, I’m pretty sure not sacrificing children is one of them. See: Jonathan Haidt
Okay, so I’ll ask again: why couldn’t the humans real preference be to not sacrifice the children? [...] In the latter case, you still haven’t explained why giving up the children is winning, and not doing so is not winning.
It seems like you are seeing my replies as soldier-arguments for the object-level question about the sacrifice of children, stumped on a particular conclusion that sacrificing children is right, while I’m merely giving opinion-neutral meta-comments about the semantics of such opinions. (I’m not sure I’m reading this right.)
You can’t decide your preference, preference is not what you actually do, it is what you should do.
You haven’t really elucidated this. You’re either pulling an ought out of nowhere, or you’re saying “preference is what you should do if you want to win”.
Preference defines what constitutes winning, your actions rank high in the preference order if they determine the world high in preference order. Preference can’t be reduced to winning or actions, as these all are the sides of the same structure.
It seems like you are seeing my replies as soldier-arguments for the object-level question about the sacrifice of children, stumped on a particular conclusion that sacrificing children is right, while I’m merely giving opinion-neutral meta-comments about the semantics of such opinions. (I’m not sure I’m reading this right.)
...so you’re NOT attempting to respond to my original question? My original question was “what’s irrational about not sacrificing the children?”
There is nothing intrinsically irrational about any action, rationality or irrationality depends on preference, which is the point I was trying to communicate. Any question about “rationality” of a decision is a question about correctness of preference-optimization. So, my reply to your original question is that the question is ill-posed, and the content of the reply was explanation as to why.
Okay, that’s fine. So you’ll agree that the various people—who were saying that the decision made in the show was the rational route—these people were speaking (at least somewhat) improperly?
You can’t decide your preference, preference is not what you actually do, it is what you should do, and it’s encoded in your decision-making capabilities in a nontrivial way, so that you aren’t necessarily capable of seeing what it is.
Compare preference to a solution to an equation: you can see the equation, you can take it apart on the constituent terms, but its solution is nowhere to be found explicitly. Yet this solution is (say) uniquely defined by the equation, and approximate methods for solving the equation (analogized to the actual decisions) tend to give their results in the general ballpark of the exact solution.
You’ve lost me.
The analogy in the next paragraph was meant to clarify. Do you see the analogy?
A person in this analogy is an equations together with an algorithm for approximately solving that equation. Decisions that the person makes are the approximate solutions, while preference is the exact solution hidden in the equation that the person can’t solve exactly. The decision algorithm tries to make decisions as close to the exact solution as it can. The exact solution is what the person should do, while the output of the approximate algorithm is what the person actually does.
I suppose I’m questioning the validity of the analogy: equations are by nature descriptive, while what one ought to do is prescriptive. Are you familiar with the Is-Ought problem?
jwdink, I don’t think Vladimir Nesov is making an Is-Ought error. Think of this: You have values (preferences, desired ends, emotional “impulses” or whatever) which are a physical part of your nature. Everything you decide to do, you do because you Want to. If you refuse to acknowledge any criteria for behavior as valuable to you, you’re saying that what feels valuable to you isn’t valuable to you. This is a contradiction!
An Is-Ought problem arises when you attempt to derive a Then without an If. Here, the If is given: If you value what you value, then you should do what is right in accordance with your values.
But there seemed to be some suggestion that an avoidance of sacrificing the children, even to the risk of everyone’s lives was a “less rational” value. If it’s a value, it’s a value… how do you call certain values invalid, or not “real” preferences?
I missed where Vladimir made that suggestion, though I’m sure others have. You can have an irrational value, if it’s really a means and not an end (which is another value), but you don’t recognize that, and call the means a value itself. Means to an end can of course be evaluated as rational. If anyone made the suggestion you mention, they probably presumed a single “basic” value of preserving lives, and considered the method of deciding to be a means, but denoted as a value.
(Of course, a value can be both a means and an end, which presents fun new complications...)
I agree generally that this is what an irrational value would mean. However, the presiding implicit assumption was that the utilitarian ends were the correct, and therefore the presiding explicit assumption (or at least, I thought it was presiding… now I can’t seem to get anyone to defend it, so maybe not) was that therefore the most efficient means to these particular ends were the most rational.
Maybe I was misunderstanding the presiding assumption, though. It was just stuff like this:
Or this, in response to a call to “dignity”:
I think I hear you, but this comment is way confusing.
Haha, we must have very different criteria for “confusing.” I found that post very clear, and I’ve struggled quite a bit with most of your posts. No offense meant, of course: I’m just not very versed in the LW vernacular.
My comments can be confusing, or difficult to get over the wider inferential gaps. In this case I meant that nickernst’s comment could just be expressed much more clearly.
The problem is a confusion. Human preference is something implemented in the very real human brain.
That’s not a particularly helpful or elucidating response. Can you flesh out your position? It’s impossible to tell what it is based on the paltry statements you’ve provided. Are you asserting that the “equation” or “hidden preference” is the same for all humans, or ought to be the same, and therefore is something objective/rational?
Preference of a given human is defined by their brain, and can be somewhat different from person to person, but not too much. There is nothing “objective” about this preference, but for each person there is one true preference that is their own, and same could be said for humanity as a whole, with the whole planet defining its preference, instead of just one brain. The focus on the brain isn’t very accurate though, since environment plays its part as well.
I can’t do justice to the centuries-old problem with a few words, but the idea is more or less this. Whatever the concept of “preference” means, when the human philosophers talk about it, their words are caused by something in the world: “preference” must be either a mechanism in their brain, a name of their confusion, or something else. It’s not epiphenomenal. Searching for the “ought” in the world outside human minds is more or less a guaranteed failure, especially if the answer is expected to be found explicitly, as an exemplar of perfection rather than evidence about what perfection is, to be interpreted in nontrivial way. The history of failure to find an answer while looking in the wrong place doesn’t prove that the answer is nowhere to be found, that there is now positive knowledge about the absence of the answer is the world.
Okay, so I’ll ask again: why couldn’t the humans real preference be to not sacrifice the children? Remember, you said:
You haven’t really elucidated this. You’re either pulling an ought out of nowhere, or you’re saying “preference is what you should do if you want to win”. In the latter case, you still haven’t explained why giving up the children is winning, and not doing so is not winning.
And the link you gave doesn’t help at all, since, if we’re going to be looking at moral impulses common to all cultures and humans, I’m pretty sure not sacrificing children is one of them. See: Jonathan Haidt
It seems like you are seeing my replies as soldier-arguments for the object-level question about the sacrifice of children, stumped on a particular conclusion that sacrificing children is right, while I’m merely giving opinion-neutral meta-comments about the semantics of such opinions. (I’m not sure I’m reading this right.)
Preference defines what constitutes winning, your actions rank high in the preference order if they determine the world high in preference order. Preference can’t be reduced to winning or actions, as these all are the sides of the same structure.
...so you’re NOT attempting to respond to my original question? My original question was “what’s irrational about not sacrificing the children?”
There is nothing intrinsically irrational about any action, rationality or irrationality depends on preference, which is the point I was trying to communicate. Any question about “rationality” of a decision is a question about correctness of preference-optimization. So, my reply to your original question is that the question is ill-posed, and the content of the reply was explanation as to why.
Okay, that’s fine. So you’ll agree that the various people—who were saying that the decision made in the show was the rational route—these people were speaking (at least somewhat) improperly?