Our core values were formed in our ancestral environment. They work quite well there, and can be generalized prety well for the certain distance outside of it. But if you go too far, such as to where core concepts such as human, sentience, life, will etc become vague, they fall apart. Even the most advanced and aligned AI will not be able to effectively generalize them there, because of the nature of our values themself. They are simply were not made for those situations and do not work there.
Our core values were formed in our ancestral environment. They work quite well there, and can be generalized prety well for the certain distance outside of it. But if you go too far, such as to where core concepts such as human, sentience, life, will etc become vague, they fall apart. Even the most advanced and aligned AI will not be able to effectively generalize them there, because of the nature of our values themself. They are simply were not made for those situations and do not work there.