How certain are you that the CEV of all of humanity agrees?
Since CEV is, AFAICT, defined by the mean of all humans’ utility functions (derived from reflective equilibrium), it disagrees by definition. But CEV is not divine revelation: it’s just the best Eliezer could do at the time. As we learn more about the evaluative and social cognition underlying our “moral” judgements, I expect to be able to dissolve a priori philosophizing and build a better Friendly goal-system than CEV.
Of course, this is because I don’t believe in normative ethics in the normal sense (post-hoc consequentialist based on Peter Railton’s style of moral realism, blah blah blah), so I’m sure this will wind up a massive debate at some point. I strongly doubt the armchair philosophers will go down without a fight.
Since CEV is, AFAICT, defined by the mean of all humans’ utility functions (derived from reflective equilibrium), it disagrees by definition. But CEV is not divine revelation: it’s just the best Eliezer could do at the time. As we learn more about the evaluative and social cognition underlying our “moral” judgements, I expect to be able to dissolve a priori philosophizing and build a better Friendly goal-system than CEV.
Of course, this is because I don’t believe in normative ethics in the normal sense (post-hoc consequentialist based on Peter Railton’s style of moral realism, blah blah blah), so I’m sure this will wind up a massive debate at some point. I strongly doubt the armchair philosophers will go down without a fight.