I think it is perfectly obvious that this usage of “should” and so on is wrong. A paperclipper believes that it should make paperclips, and it means exactly the same thing by “should” that I do when I say I should not murder.
And when I say it is obvious, I mean it is obvious in the same way that it is obvious that you are using the word “hat” wrong if you use it for a coat.
I think you’re using “should” to mean “feels compelled to do.”
Yes, a paperclipper feels compelled to make paperclips, and a human feels compelled to make sentient beings happy.
But when we say “should,” we don’t just mean “whatever anyone feels compelled to do.” We say “you might drug me to make me want to kill people, but I still shouldn’t do it.”
“Should” does not refer to compelling feelings, but rather to a certain set of states of beings that we value. To say we “still shouldn’t kill people,” means it “still isn’t in harmony with happy sentient beings (plus a million other values) to kill people.”
A paperclipper wouldn’t disagree that killing people isn’t in harmony with happy sentient beings (along with a million other values), it just wouldn’t care. In other words, it wouldn’t disagree that it shouldn’t kill people, it just doesn’t care about “should;” it cares about “clipperould.”
Likewise, we wouldn’t disagree that keeping people around instead of making them into paperclips is not in harmony with maximizing paperclips, we just wouldn’t care. We know we clipperould turn people into paperclips, we just don’t care about clipperould, we care about should.
No, I am not using “should” to mean “feels...” anything (in other words, feelings have nothing to do with it.) But you are right about compulsion. The word “ought” is, in theory, just the past tense of “owe”, and what is owed is something that needs to be paid. Saying that you ought to do something, just means that you need to do it. And should is the same; that you should do it just means that there is a need for it. And need is just necessity. So it does all have to do with compulsion.
But it is not compulsion of feelings, but of a goal. And to that degree, your idea is actually correct. But you are wrong to say that the specific goal sought affects the meaning of the word. “I should do it” means that I need to do it to attain my goal. It does not say what that goal is.
I think it is perfectly obvious that this usage of “should” and so on is wrong. A paperclipper believes that it should make paperclips, and it means exactly the same thing by “should” that I do when I say I should not murder.
And when I say it is obvious, I mean it is obvious in the same way that it is obvious that you are using the word “hat” wrong if you use it for a coat.
I think you’re using “should” to mean “feels compelled to do.”
Yes, a paperclipper feels compelled to make paperclips, and a human feels compelled to make sentient beings happy.
But when we say “should,” we don’t just mean “whatever anyone feels compelled to do.” We say “you might drug me to make me want to kill people, but I still shouldn’t do it.”
“Should” does not refer to compelling feelings, but rather to a certain set of states of beings that we value. To say we “still shouldn’t kill people,” means it “still isn’t in harmony with happy sentient beings (plus a million other values) to kill people.”
A paperclipper wouldn’t disagree that killing people isn’t in harmony with happy sentient beings (along with a million other values), it just wouldn’t care. In other words, it wouldn’t disagree that it shouldn’t kill people, it just doesn’t care about “should;” it cares about “clipperould.”
Likewise, we wouldn’t disagree that keeping people around instead of making them into paperclips is not in harmony with maximizing paperclips, we just wouldn’t care. We know we clipperould turn people into paperclips, we just don’t care about clipperould, we care about should.
No, I am not using “should” to mean “feels...” anything (in other words, feelings have nothing to do with it.) But you are right about compulsion. The word “ought” is, in theory, just the past tense of “owe”, and what is owed is something that needs to be paid. Saying that you ought to do something, just means that you need to do it. And should is the same; that you should do it just means that there is a need for it. And need is just necessity. So it does all have to do with compulsion.
But it is not compulsion of feelings, but of a goal. And to that degree, your idea is actually correct. But you are wrong to say that the specific goal sought affects the meaning of the word. “I should do it” means that I need to do it to attain my goal. It does not say what that goal is.