Eliezer’s one-place function is exactly infallible, because he defines “right” as its output.
I misunderstood some of Eliezer’s notation. I now take his function to be an extrapolation of his volition rather than anyone else’s. I don’t think this weakens my point: if there were a rock somewhere with a lookup table for this function written on it, Eliezer should always follow the rock rather than his own insights (and according to Eliezer everyone else should too), and this remains true even if there is no such rock.
Furthermore, the morality function is based on extrapolated volition. Someone who has only considered one point of view on various moral questions will disagree with their extrapolated (completely knowledgable, completely wise) volition in certain predictable ways. That’s exactly what I mean by a “twist.”
Nick,
Eliezer’s one-place function is exactly infallible, because he defines “right” as its output.
I misunderstood some of Eliezer’s notation. I now take his function to be an extrapolation of his volition rather than anyone else’s. I don’t think this weakens my point: if there were a rock somewhere with a lookup table for this function written on it, Eliezer should always follow the rock rather than his own insights (and according to Eliezer everyone else should too), and this remains true even if there is no such rock.
Furthermore, the morality function is based on extrapolated volition. Someone who has only considered one point of view on various moral questions will disagree with their extrapolated (completely knowledgable, completely wise) volition in certain predictable ways. That’s exactly what I mean by a “twist.”