I think we are largely agreed as to facts and disagree only on whether it’s better to call Eliezer’s theory, which is intermediate between many realist theories and many non-realist theories, “realist” or “non-realist”.
I’m not sure, though, that someone who says that “this is good” = “I want to do this” is really a typical non-realist. My notion of a typical non-realist—typical, I mean, among people who’ve actually thought seriously about this stuff—is somewhat nearer to Eliezer’s position than that.
Anyway, the reason why I class Eliezer’s position as non-realist is that the distinction between Eliezer’s position and that of many (other?) non-realists is purely terminological—he agrees that there are all these various value systems, and that if ours seems special to us that’s because it’s ours rather than because of some agent-independent feature of the universe that picks ours out in preference to others, but he wants to use words like “good” to refer to one particular value system—whereas the distinction between his position and that of most (other?) realists goes beyond terminology: they say that the value system they regard as real is actually built into the fabric of reality in some way that goes beyond the mere fact that it’s our (or their) value system.
I think he wants a system which works like realism, in that there are definite answers to ethical questions (“fixed”, “frozen”) ,but without spookiness.
Yudkowsky,’s theory entails the same problem as relativism: if morality is whatever people value, and if what people happen to value is intuitively immoral , slavery, torture,whatever, then there’s no fixed standard of morality. The label “moral” has been placed on a moving target. (Standard relativism usually has this problem synchronously
, ie different communities are said to have different but equally valid moralities at the same time, but it makes little difference if you are asserting that the global community has different but equally valid moralities at different times)
You can avoid the problems of relativism by setting up an external standard, and there are many theories of that type, but they tend to have the problem that the external standard is not naturalistic....God’s commands, the Form of the good, and so on. I think Yudkowsky wants a theory that is non arbitrary and also naturalistic. I don’t think he arrives a single theory that does both. If the Moral Equation is just a label for human intuition, then it ssuffers from all the vagaries of labeling values as moral, the original theory. If the Moral Equation is something ideal and abstract, why can’t aliens partake?
I think we are largely agreed as to facts and disagree only on whether it’s better to call Eliezer’s theory, which is intermediate between many realist theories and many non-realist theories, “realist” or “non-realist”.
I’m not sure, though, that someone who says that “this is good” = “I want to do this” is really a typical non-realist. My notion of a typical non-realist—typical, I mean, among people who’ve actually thought seriously about this stuff—is somewhat nearer to Eliezer’s position than that.
Anyway, the reason why I class Eliezer’s position as non-realist is that the distinction between Eliezer’s position and that of many (other?) non-realists is purely terminological—he agrees that there are all these various value systems, and that if ours seems special to us that’s because it’s ours rather than because of some agent-independent feature of the universe that picks ours out in preference to others, but he wants to use words like “good” to refer to one particular value system—whereas the distinction between his position and that of most (other?) realists goes beyond terminology: they say that the value system they regard as real is actually built into the fabric of reality in some way that goes beyond the mere fact that it’s our (or their) value system.
You may weight these differences differently.
I think he wants a system which works like realism, in that there are definite answers to ethical questions (“fixed”, “frozen”) ,but without spookiness.
Yudkowsky,’s theory entails the same problem as relativism: if morality is whatever people value, and if what people happen to value is intuitively immoral , slavery, torture,whatever, then there’s no fixed standard of morality. The label “moral” has been placed on a moving target. (Standard relativism usually has this problem synchronously , ie different communities are said to have different but equally valid moralities at the same time, but it makes little difference if you are asserting that the global community has different but equally valid moralities at different times)
You can avoid the problems of relativism by setting up an external standard, and there are many theories of that type, but they tend to have the problem that the external standard is not naturalistic....God’s commands, the Form of the good, and so on. I think Yudkowsky wants a theory that is non arbitrary and also naturalistic. I don’t think he arrives a single theory that does both. If the Moral Equation is just a label for human intuition, then it ssuffers from all the vagaries of labeling values as moral, the original theory. If the Moral Equation is something ideal and abstract, why can’t aliens partake?
I agree.