CEV is not preference utilitarianism, or any other first-order ethical theory. Rather, preference utilitarianism is the sort of thing that might be CEV’s output.
Obviously CEV isn’t identical to preference utilitarianism, but CEV and preference utilitarianism have the following principles in common, which the hack exploits:
Give people what they want, instead of what you think is good for them.
If different people want different things, give each individual equal weight.
It seems clear that Eliezer got these ideas from preference utilitarianism, and they share some of the same flaws as a result.
CEV is not preference utilitarianism, or any other first-order ethical theory. Rather, preference utilitarianism is the sort of thing that might be CEV’s output.
Obviously CEV isn’t identical to preference utilitarianism, but CEV and preference utilitarianism have the following principles in common, which the hack exploits:
Give people what they want, instead of what you think is good for them.
If different people want different things, give each individual equal weight.
It seems clear that Eliezer got these ideas from preference utilitarianism, and they share some of the same flaws as a result.