In the usages he has made EY actually seems to say there is a “should”, which we would describe as should. For other preferences he has suggested would_want. So if John wants to murder people he should not murder people but would_want to murder them. (But that is just his particular semantics, the actual advocated behavior is as you describe it.)
I don’t remember the would_want semantics anywhere in EY’s writings, but I see the appeal—especially given how my discussion with Peterdjones is going,
I don’t remember the would_want semantics anywhere in EY’s writings
It was in a past conversation on the subject of what Eliezer means by “should” and related terms. That was the answer he gave in response to the explicit question. In actual writings there hasn’t been a particular need to refer concisely to the morality of other agents independently of their actual preferences. When describing Baby Eaters, for example, natural language worked just fine.
I don’t remember the would_want semantics anywhere in EY’s writings, but I see the appeal—especially given how my discussion with Peterdjones is going,
It was in a past conversation on the subject of what Eliezer means by “should” and related terms. That was the answer he gave in response to the explicit question. In actual writings there hasn’t been a particular need to refer concisely to the morality of other agents independently of their actual preferences. When describing Baby Eaters, for example, natural language worked just fine.