To clarify, I meant the AI is unlikely to have it by default (being able to perfectly simulate a person does not in itself require having empathy as part of the reward function).
If we try to hardcode it, Goodhart’s curse seems relevant: https://arbital.com/p/goodharts_curse/
But note that Reward is not the optimization target
To clarify, I meant the AI is unlikely to have it by default (being able to perfectly simulate a person does not in itself require having empathy as part of the reward function).
If we try to hardcode it, Goodhart’s curse seems relevant: https://arbital.com/p/goodharts_curse/
But note that Reward is not the optimization target