Rephrased, this knowledge is just the notion that you instantiate some computation instead of not doing (or being) anything. This way, my confidence in its truth is very high, although of course not 1.
We know we instantiate some computation. But it’s a pre-theoretic datum that we don’t know that our fundamental values are right. So EY’s theory misdescribes the concept of rightness.
(This is basically a variation on Moore’s Open Question Argument.)
I’d be okay with a strong AI that correctly followed my values, regardless of whether they’re “right” by any other criterion.
If you think you wouldn’t be okay with such an AI, I suspect the most likely explanation is that you’re confused about the concept of “your values”. Namely, if you yearn to discover some simple external formula like the categorical imperative and then enact the outcomes prescribed by that formula, then that’s just another fact about your personal makeup that has to be taken into account by the AI.
And if you agree that you would be okay with such an AI, that means Eliezer’s metaethics is adequate for its stated goal (creating friendly AI), whatever other theoretical drawbacks it might have.
Rephrased, this knowledge is just the notion that you instantiate some computation instead of not doing (or being) anything. This way, my confidence in its truth is very high, although of course not 1.
We know we instantiate some computation. But it’s a pre-theoretic datum that we don’t know that our fundamental values are right. So EY’s theory misdescribes the concept of rightness.
(This is basically a variation on Moore’s Open Question Argument.)
Huh?
I’d be okay with a strong AI that correctly followed my values, regardless of whether they’re “right” by any other criterion.
If you think you wouldn’t be okay with such an AI, I suspect the most likely explanation is that you’re confused about the concept of “your values”. Namely, if you yearn to discover some simple external formula like the categorical imperative and then enact the outcomes prescribed by that formula, then that’s just another fact about your personal makeup that has to be taken into account by the AI.
And if you agree that you would be okay with such an AI, that means Eliezer’s metaethics is adequate for its stated goal (creating friendly AI), whatever other theoretical drawbacks it might have.