Also, it appears from the comment posted below and earlier that this mysterious missed point is accessible to, for example, Nick Tarleton.
he defines it (as I understand) in terms of an objective computation that happens to be instantiated by humans
No, I understand that your long list wouldn’t change if humanity itself changed, that if I altered every human to like eating babies, that wouldn’t make babyeating right, in the Eliezer world.
“the rightness computation that humanity just happens to instantiate” is different from “whatever computation humanity instantiates”
No, I understand that your long list wouldn’t change if humanity itself changed, that if I altered every human to like eating babies, that wouldn’t make babyeating right, in the Eliezer world.
“the rightness computation that humanity just happens to instantiate” is different from “whatever computation humanity instantiates”
Is there something else I am missing?