Are you sure Eliezer does equate reflective consistency with alignment with what-he-calls-”right”? Because my recollection is that he doesn’t claim either (1) that a reflectively consistent alien mind need have values at all like what he calls right, or (2) that any individual human being, if made reflectively consistent, would necessarily end up with values much like what he calls right.
(Unless I’m awfully confused, denial of (1) is an important element in his thinking.)
I think he is defining “right” to mean something along the lines of “in line with the CEV of present-day humanity”. Maybe that’s a sensible way to use the word, maybe not (for what it’s worth, I incline towards “not”) but it isn’t the same thing as identifying “right” with “reflectively consistent”, and it doesn’t lead to a risk of confusion if the two turn out to have different referents (because they can’t).
Are you sure Eliezer does equate reflective consistency with alignment with what-he-calls-”right”? Because my recollection is that he doesn’t claim either (1) that a reflectively consistent alien mind need have values at all like what he calls right, or (2) that any individual human being, if made reflectively consistent, would necessarily end up with values much like what he calls right.
(Unless I’m awfully confused, denial of (1) is an important element in his thinking.)
I think he is defining “right” to mean something along the lines of “in line with the CEV of present-day humanity”. Maybe that’s a sensible way to use the word, maybe not (for what it’s worth, I incline towards “not”) but it isn’t the same thing as identifying “right” with “reflectively consistent”, and it doesn’t lead to a risk of confusion if the two turn out to have different referents (because they can’t).