I think this is not the subject matter that most people are talking about when they talk about morality.
True, as long as they’re talking about the stuff that is implied by their terminal values.
However, when they start talking about the stuff that is implied by other people’s (or aliens’, or AIs’) terminal values, the meaning they attach to the word ‘morality’ is a lot closer to the one I’m proposing. They might say things like, “Well, female genital mutilation is moral to Sudanese people. Um, I mean, errr, uh...”, and then they’re really confused. This confusion would vanish (or at least, would be more likely to vanish) if they were forced to say, “Well, female genital mutilation is Sudanese-moral but me-immoral.”
Ideally, to avoid all confusion we should get rid of the word morality completely, and have everyone speak in terms of goals and desires instead.
Agreed. If it happened that there were only a few different sets of terminal values in existence, then I would be OK with assigning different words to the pursuit of those different sets. One of those words could be ‘moral’. However, as is, the set of all terminal values represented by humans is too fractured and varied.
A large chunk of the list Eliezer provides in the above comment probably is nearly universal to humanity, but the entire list is not, and there are certainly many disputes on the relative ordering (especially as to what is on top).
True, as long as they’re talking about the stuff that is implied by their terminal values.
However, when they start talking about the stuff that is implied by other people’s (or aliens’, or AIs’) terminal values, the meaning they attach to the word ‘morality’ is a lot closer to the one I’m proposing. They might say things like, “Well, female genital mutilation is moral to Sudanese people. Um, I mean, errr, uh...”, and then they’re really confused. This confusion would vanish (or at least, would be more likely to vanish) if they were forced to say, “Well, female genital mutilation is Sudanese-moral but me-immoral.”
Ideally, to avoid all confusion we should get rid of the word morality completely, and have everyone speak in terms of goals and desires instead.
Agreed. If it happened that there were only a few different sets of terminal values in existence, then I would be OK with assigning different words to the pursuit of those different sets. One of those words could be ‘moral’. However, as is, the set of all terminal values represented by humans is too fractured and varied.
A large chunk of the list Eliezer provides in the above comment probably is nearly universal to humanity, but the entire list is not, and there are certainly many disputes on the relative ordering (especially as to what is on top).