The reason Eliezer’s views are commonly mistaken for relativism in the manner you describe is because most people do not have a good grasp on the difference between sense and reference(a difference that, to be fair, doesn’t seem to be well explained anywhere). To elaborate:
“Define good and bad and so forth in terms of what human beings, in fact, value” sounds like saying that goodness depends on human values. This is the definition you get if you say “let ‘good’ mean ‘human values’”. But the actual idea is meant to be more analogous to this: assuming for the sake of argument that humans value cake, define “good” to mean cake. Obviously, under that definition, “cake is always good regardless of what humans value” is true. In that case “good” is a rigid designator for cake.
The difference is that “good” and “human values” are not synonymous. But they refer to the same thing, when you fully dereference them, namely {happiness, fun and so forth}. This is the difference between sense and reference, and it’s why it is necessary to understand rigid designators.
Here is my question: Why bother with the middle man? No one can actually define good and everyone is constantly checking with ‘human values’ to see what it says! Assuming the universe runs on math and humans share attitudes about some things obviously there is some platonic entity which precisely describes human values (assuming there isn’t too much contradiction) and can be called “good”. But it doesn’t seem especially parsimonious to reify that concept. Why add it to our ontology?
It’s just semantics in a sense: but there is a reason we don’t multiply entities unnecessarily.
Well, if you valued cake you’d want a way to talk about cake and efficiently distinguish cakes from non-cakes—-and especially with regards to planning, to distinguish plans that lead to cake from plans that do not. When you talk about cake there isn’t really any reification of “the platonic form of cake” going on; “cake” is just a convenient word for a certain kind of confection.
The motivation for humans having a word for goodness is the same.
I don’t necessarily have a problem with using the word “good” so long as everyone understands it isn’t something out there in the world that we’ve discovered—that it’s a creation of our minds, words and behavior—like cake. This is a problem because most of the world doesn’t think that. A lot of times it doesn’t seem like Less Wrong thinks that (but I’m beginning to think that is just non-standard terminology).
Yeah, a lot of the Metaethics Sequence seems to be trying to get to this point.
For my part, it seems easier to just stop using words like “good” if we believe they are likely to be misunderstood, rather than devoting a lot of energy to convincing everyone that they should mean something different by the word (or that the word really means something different from what they think it means, or whatever).
I’m content to say that we value what we currently value, because we currently value it, and asking whether that’s good or not is asking an empty question.
Of course, I do understand the rhetorical value of getting to claim that our AI does good, rather than “merely” claiming that it implements what we currently value.
I’m content to say that we value what we currently value, because we currently value it, and asking whether that’s good or not is asking an empty question.
I am content to say the question is not empty, and if you assumptions lead you to suppose it is, then your assumptions need to be questioned.
Yes, sorry, I wasn’t clear enough about that. No, let me go further; what I wrote was downright misleading. This is why I shouldn’t write Less Wrong comments on a tablet where I am too strongly incentivized to make them brief :-). I endorse your description of Eliezer’s position.
This is the definition you get if you say “let ‘good’ mean ‘human values’”. But the actual idea is meant to be more analogous to this: assuming for the sake of argument that humans value cake, define “good” to mean cake. Obviously, under that definition, “cake is always good regardless of what humans value” is true. In that case “good” is a rigid designator for cake.
Why is cake a referent of good?
The difference is that “good” and “human values” are not synonymous. But they refer to the same thing, when you fully dereference them, namely {happiness, fun and so forth}. This is the difference between sense and reference, and it’s why it is necessary to understand rigid designators.
And what happened to the normativity of Good? Why does it appear to make sense to wonder if we are valuing the right things, when Good is just whatever we value?
ADDED:
The reason Eliezer’s views are commonly mistaken for relativism in the manner you describe is because most people do not have a good grasp on the difference between sense and reference(a difference that, to be fair, doesn’t seem to be well explained anywhere).
I don’t see the S/R difference is relevant to relativism. If the referents of “good” vary with the mental contents of the person saying “good”, that is relativism/subjectivism. (That the values referenced are ultimately physical does not affect that: relativism is an epistemological claim, not a metaphysical one).
Why does it appear to make sense to wonder if we are valuing the right things
For a start, the fact that some things seem to make sense is not a oracular window unto philosophical truth. Anything that we are unsure about will seem as if it could go either way, even if one of the options is in fact logically necessary or empirically true. That’s the point of being unsure (example: the Riemann conjecture).
At the object level, no-one knows in full detail exactly what they mean by “good”, or the detailed contents of their own values. So trying to test “my values are good” by direct comparison, so to speak, is a highly nontrivial (read: impossible) exercise. Figuring out based on things like “wanting to do the right thing” that “good” and “human values” refer to the same thing while not being synonymous is another nontrivial exercise.
I don’t see the S/R difference is relevant to relativism. If the referents of “good” vary with the mental contents of the person saying “good”, that is relativism/subjectivism. (That the values referenced are ultimately physical does not affect that: relativism is an epistemological claim, not a metaphysical one).
To me, the fact that you don’t understand is evidence the difference matters. Unless you’re saying that “relativism” is just the statement that people on different planets speak different languages, in which case, “no shit” as the French say.
I was wondering how one knows what the referents of good are when one doesn’t know the sense.
I didn’t claim that anything was anoracular window. But note that things you believe in, such as an external world, can just as glibly be dismissed as illusion.
Why does it appear to make sense to wonder if we are valuing the right things, when Good is just whatever we value?
We are in the habit of (and reinforced for) asking certain questions about actual real-world things. “Is the food I’m eating good food?” “Is the wood I’m building my house out of good wood?” “Is the exercise program I’m starting a good exercise program?” Etc. In each case, we have some notion of what we mean by the question that grounds out in some notion of our values… that is, in what we want food, housing materials, and exercise programs to achieve.
We continue to apply that habitual formula even in cases where we’re not very clear what those values are, what we want those things to achieve. “Is democracy a good political system?” is a compelling-sounding question even for people who lack a clear understanding of what their political values are; “Is Christianity a good religion?” feels like a meaningful question to many people who don’t have a clear notion of what they want a religion to achieve.
That we continue to apply the same formula to get the question “Are the values I’m using good values?” should not surprise us; I would expect us to ask it and for it to feel meaningful whether it actually makes sense or not.
But when you ask a question and someone provides an answer you don’t like, showing why that answer is wrong can sometimes be more effective than simply asserting that you don’t buy it.
And any theory can be made to fail if I am allowed to demand that it explain things that don’t actually exist.
So it seems to matter whether the thing I’m dismissing exists or not.
Regardless, all of this is a tangent from my point.
You asked “Why does it appear to make sense to wonder if we are valuing the right things?” as a rhetorical question, as a way of arguing that it appears to make sense because it does make sense, because the question of whether our values are right is non-empty. My point is that this is not actually why it appears to make sense; it would appear to make sense even if the question of whether our values are right were empty.
That is not proof that the question is empty, of course. All it demonstrates is that one of your arguments in defense of its non-emptiness is flawed.
You will probably do better to accept that and marshall your remaining argument-soldiers to a victorious campaign on other fronts.
Rigid designators
The reason Eliezer’s views are commonly mistaken for relativism in the manner you describe is because most people do not have a good grasp on the difference between sense and reference(a difference that, to be fair, doesn’t seem to be well explained anywhere). To elaborate:
“Define good and bad and so forth in terms of what human beings, in fact, value” sounds like saying that goodness depends on human values. This is the definition you get if you say “let ‘good’ mean ‘human values’”. But the actual idea is meant to be more analogous to this: assuming for the sake of argument that humans value cake, define “good” to mean cake. Obviously, under that definition, “cake is always good regardless of what humans value” is true. In that case “good” is a rigid designator for cake.
The difference is that “good” and “human values” are not synonymous. But they refer to the same thing, when you fully dereference them, namely {happiness, fun and so forth}. This is the difference between sense and reference, and it’s why it is necessary to understand rigid designators.
This is an excellent description of the argument.
Here is my question: Why bother with the middle man? No one can actually define good and everyone is constantly checking with ‘human values’ to see what it says! Assuming the universe runs on math and humans share attitudes about some things obviously there is some platonic entity which precisely describes human values (assuming there isn’t too much contradiction) and can be called “good”. But it doesn’t seem especially parsimonious to reify that concept. Why add it to our ontology?
It’s just semantics in a sense: but there is a reason we don’t multiply entities unnecessarily.
Well, if you valued cake you’d want a way to talk about cake and efficiently distinguish cakes from non-cakes—-and especially with regards to planning, to distinguish plans that lead to cake from plans that do not. When you talk about cake there isn’t really any reification of “the platonic form of cake” going on; “cake” is just a convenient word for a certain kind of confection.
The motivation for humans having a word for goodness is the same.
I don’t necessarily have a problem with using the word “good” so long as everyone understands it isn’t something out there in the world that we’ve discovered—that it’s a creation of our minds, words and behavior—like cake. This is a problem because most of the world doesn’t think that. A lot of times it doesn’t seem like Less Wrong thinks that (but I’m beginning to think that is just non-standard terminology).
Yeah, a lot of the Metaethics Sequence seems to be trying to get to this point.
For my part, it seems easier to just stop using words like “good” if we believe they are likely to be misunderstood, rather than devoting a lot of energy to convincing everyone that they should mean something different by the word (or that the word really means something different from what they think it means, or whatever).
I’m content to say that we value what we currently value, because we currently value it, and asking whether that’s good or not is asking an empty question.
Of course, I do understand the rhetorical value of getting to claim that our AI does good, rather than “merely” claiming that it implements what we currently value.
.
I am content to say the question is not empty, and if you assumptions lead you to suppose it is, then your assumptions need to be questioned.
You seem to believe that I have arrived at my current position primarily via unquestioned assumptions.
What makes you conclude that?
Yes, sorry, I wasn’t clear enough about that. No, let me go further; what I wrote was downright misleading. This is why I shouldn’t write Less Wrong comments on a tablet where I am too strongly incentivized to make them brief :-). I endorse your description of Eliezer’s position.
Why is cake a referent of good?
And what happened to the normativity of Good? Why does it appear to make sense to wonder if we are valuing the right things, when Good is just whatever we value?
ADDED:
I don’t see the S/R difference is relevant to relativism. If the referents of “good” vary with the mental contents of the person saying “good”, that is relativism/subjectivism. (That the values referenced are ultimately physical does not affect that: relativism is an epistemological claim, not a metaphysical one).
Why do we have words that mean things at all?
For a start, the fact that some things seem to make sense is not a oracular window unto philosophical truth. Anything that we are unsure about will seem as if it could go either way, even if one of the options is in fact logically necessary or empirically true. That’s the point of being unsure (example: the Riemann conjecture).
At the object level, no-one knows in full detail exactly what they mean by “good”, or the detailed contents of their own values. So trying to test “my values are good” by direct comparison, so to speak, is a highly nontrivial (read: impossible) exercise. Figuring out based on things like “wanting to do the right thing” that “good” and “human values” refer to the same thing while not being synonymous is another nontrivial exercise.
To me, the fact that you don’t understand is evidence the difference matters. Unless you’re saying that “relativism” is just the statement that people on different planets speak different languages, in which case, “no shit” as the French say.
I was wondering how one knows what the referents of good are when one doesn’t know the sense.
I didn’t claim that anything was anoracular window. But note that things you believe in, such as an external world, can just as glibly be dismissed as illusion.
We are in the habit of (and reinforced for) asking certain questions about actual real-world things. “Is the food I’m eating good food?” “Is the wood I’m building my house out of good wood?” “Is the exercise program I’m starting a good exercise program?” Etc. In each case, we have some notion of what we mean by the question that grounds out in some notion of our values… that is, in what we want food, housing materials, and exercise programs to achieve.
We continue to apply that habitual formula even in cases where we’re not very clear what those values are, what we want those things to achieve. “Is democracy a good political system?” is a compelling-sounding question even for people who lack a clear understanding of what their political values are; “Is Christianity a good religion?” feels like a meaningful question to many people who don’t have a clear notion of what they want a religion to achieve.
That we continue to apply the same formula to get the question “Are the values I’m using good values?” should not surprise us; I would expect us to ask it and for it to feel meaningful whether it actually makes sense or not.
You can argue that the things your theory can’t explain are non-issues. I don’t have to buy that,
You certainly don’t have to buy it, that’s true.
But when you ask a question and someone provides an answer you don’t like, showing why that answer is wrong can sometimes be more effective than simply asserting that you don’t buy it.
The problem is a kind of quodlibet. Any inadequate theory can be made to work if one is allowed to dismiss whatever the theory can’t explain.
Sure, I agree.
And any theory can be made to fail if I am allowed to demand that it explain things that don’t actually exist.
So it seems to matter whether the thing I’m dismissing exists or not.
Regardless, all of this is a tangent from my point.
You asked “Why does it appear to make sense to wonder if we are valuing the right things?” as a rhetorical question, as a way of arguing that it appears to make sense because it does make sense, because the question of whether our values are right is non-empty. My point is that this is not actually why it appears to make sense; it would appear to make sense even if the question of whether our values are right were empty.
That is not proof that the question is empty, of course. All it demonstrates is that one of your arguments in defense of its non-emptiness is flawed.
You will probably do better to accept that and marshall your remaining argument-soldiers to a victorious campaign on other fronts.
Non-emptiness is no more flawed than emptiness. The Open Question remains open.
This is a nonsequitor. My claim was about a specific argument.