So you believe that the word morality is a two-place word and means what an agent would want to do under certain circumstances? What word do you use to means what actually ought to be done? The particular thing that you and, to a large degree all humans would want to do under specified circumstances? Or do you believe there isn’t anything that should be done other than what whatever agents exist want? Please note that that position is also a statement about what the universe ought to look like.
Yes, morality is a two-place word—the evaluation function of whether an action is moral has two inputs: agent, action. “Agent” can be replaced by anything that conceivably has agency, so morality can be considered system-dependent, where systems include social groups and all humanity, etc.
I wouldn’t say morality is what the agent wants to do, but is what the agent ought to do, given its preferences. So I think I am still using it in the usual sense.
What word do you use to means what actually ought to be done? The particular thing that you and, to a large degree all humans would want to do under specified circumstances?
I can talk about what I ought to do, but it seems to me I can’t talk about what another agent ought to do outside their system of preferences. If I had their preferences, I ought to do what they ought to do. If they had my preferences, they ought to do what I ought to do. But to consider what they ought to do, with some mixture of preferences, isn’t incoherent.
I can have a preference for what another agent does, of course, but this is different than asserting a morality. For example, if they don’t do what I think is moral, I’m not morally culpable. I don’t have their agency.
As far as I can tell, we don’t disagree on any matter of fact. I agree that we can only optimize our own actions. I agree that other agents won’t necessarily find our moral arguments persuasive. I just don’t agree that the words moral and ought should be used the way you do.
To the greater LW community: Is there some way we can come up with standard terminology for this sort of thing? I myself have moved toward using the terminology used by Eliezer, but not everyone has. Are there severe objections to his terminology and if so, are there any other terminologies you think we should adopt as standard?
So you believe that the word morality is a two-place word and means what an agent would want to do under certain circumstances? What word do you use to means what actually ought to be done? The particular thing that you and, to a large degree all humans would want to do under specified circumstances? Or do you believe there isn’t anything that should be done other than what whatever agents exist want? Please note that that position is also a statement about what the universe ought to look like.
Yes, morality is a two-place word—the evaluation function of whether an action is moral has two inputs: agent, action. “Agent” can be replaced by anything that conceivably has agency, so morality can be considered system-dependent, where systems include social groups and all humanity, etc.
I wouldn’t say morality is what the agent wants to do, but is what the agent ought to do, given its preferences. So I think I am still using it in the usual sense.
I can talk about what I ought to do, but it seems to me I can’t talk about what another agent ought to do outside their system of preferences. If I had their preferences, I ought to do what they ought to do. If they had my preferences, they ought to do what I ought to do. But to consider what they ought to do, with some mixture of preferences, isn’t incoherent.
I can have a preference for what another agent does, of course, but this is different than asserting a morality. For example, if they don’t do what I think is moral, I’m not morally culpable. I don’t have their agency.
As far as I can tell, we don’t disagree on any matter of fact. I agree that we can only optimize our own actions. I agree that other agents won’t necessarily find our moral arguments persuasive. I just don’t agree that the words moral and ought should be used the way you do.
To the greater LW community: Is there some way we can come up with standard terminology for this sort of thing? I myself have moved toward using the terminology used by Eliezer, but not everyone has. Are there severe objections to his terminology and if so, are there any other terminologies you think we should adopt as standard?