I’m a moral cognitivist too but I’m becoming quite puzzled as to what truth-conditions you think “should” statements have. Maybe it would help if you said which of these you think are true statements.
1) Eliezer Yudkowsky should not kill babies.
2) Babyeating aliens should not kill babies.
3) Sharks should not kill babies.
4) Volcanoes should not kill babies.
5) Should not kill babies. (sic)
The meaning of “should not” in 2 through 5 are intended to be the same as the common usage of the words in 1.
Technically, you would need to include a caveat in all of those like, “unless to do so would advance paperclip production” but I assume that’s what you meant.
The meaning of “should not” in 2 through 5 are intended to be the same as the common usage of the words in 1.
I don’t think there is one common usage of the word “should”.
(ETA: I asked the nearest three people if “volcanoes shouldn’t kill people” is true, false, or neither, assuming that “people shouldn’t kill people” is true or false so moral non-realism wasn’t an issue. One said true, two said neither.)
Here’s my guess at one type of situation Eliezer might be
thinking of when calling proposition B false: It is
rational (let us stipulate) for a paperclip maximizer to
turn all the matter in the solar system into computronium in
order to compute ways to maximize paperclips, but “should”
does not apply to paperclip maximizers.
EDIT: If I were picking nits, I would say, “‘Should’ does apply to paperclip maximizers—it is rational for X to make paperclips but it should not do so—however, paperclip maximizers don’t care and so it is pointless to talk about what they should do.” But the overall intent of the statement is correct—I disagree with its intent in neither anticipation nor morals—and in such cases I usually just say “Correct”. In this case I suppose that wasn’t the best policy, but it is my usual policy.
I’m a moral cognitivist too but I’m becoming quite puzzled as to what truth-conditions you think “should” statements have. Maybe it would help if you said which of these you think are true statements.
1) Eliezer Yudkowsky should not kill babies.
2) Babyeating aliens should not kill babies.
3) Sharks should not kill babies.
4) Volcanoes should not kill babies.
5) Should not kill babies. (sic)
The meaning of “should not” in 2 through 5 are intended to be the same as the common usage of the words in 1.
Technically, you would need to include a caveat in all of those like, “unless to do so would advance paperclip production” but I assume that’s what you meant.
I don’t think there is one common usage of the word “should”.
(ETA: I asked the nearest three people if “volcanoes shouldn’t kill people” is true, false, or neither, assuming that “people shouldn’t kill people” is true or false so moral non-realism wasn’t an issue. One said true, two said neither.)
I don’t think there’s one canonical common usage of the word “should”.
(I’m not sure whether to say that 2-5 are true, or that 2-4 are type errors and 5 is a syntax error.)
They all sound true to me.
Interesting, what about either of the following:
A) If X should do A, then it is rational for X to do A.
B) If it is rational for X to do A, then X should do A.
From what I understand of what Eliezer’s position:
False
False.
(If this isn’t the case then Eliezer’s ‘should’ is even more annoying than how I now understand it.)
Yep, both false.
So, just to dwell on this for a moment, there exist X and A such that (1) it is rational for X to do A and (2) X should not do A.
How do you reconcile this with “rationalists should win”? (I think I know what your response will be, but I want to make sure.)
Here’s my guess at one type of situation Eliezer might be thinking of when calling proposition B false: It is rational (let us stipulate) for a paperclip maximizer to turn all the matter in the solar system into computronium in order to compute ways to maximize paperclips, but “should” does not apply to paperclip maximizers.
Correct.
EDIT: If I were picking nits, I would say, “‘Should’ does apply to paperclip maximizers—it is rational for X to make paperclips but it should not do so—however, paperclip maximizers don’t care and so it is pointless to talk about what they should do.” But the overall intent of the statement is correct—I disagree with its intent in neither anticipation nor morals—and in such cases I usually just say “Correct”. In this case I suppose that wasn’t the best policy, but it is my usual policy.
Of course, Kant distinguished between two different meanings of “should”: the hypothetical and the categorical.
If you want to be a better Go player, you should study the games of Honinbo Shusaku.
You should pull the baby off the rail track.
This seems useful here...
False. Be consistent.