In this specific case, what ends up dominating CEV is what evolution wants, not what we want.
Possibly. It also sounds like the best part of Robert Heinlein’s Good Outcome for the future. I think we can do better—but you seem to be arguing for the claim that we can’t. Still beats paperclips, or even true orgasmium.
We can do better if we take this kind of problem in consideration. If there is too much of what Eliezer calls spread and muddle, we may end up just evolving faster. I don’t think blind faster evolution would be on top of anyone’s list of desires.
Possibly. It also sounds like the best part of Robert Heinlein’s Good Outcome for the future. I think we can do better—but you seem to be arguing for the claim that we can’t. Still beats paperclips, or even true orgasmium.
We can do better if we take this kind of problem in consideration. If there is too much of what Eliezer calls spread and muddle, we may end up just evolving faster. I don’t think blind faster evolution would be on top of anyone’s list of desires.