Note also that defining the thing-we-want-an-AI-to-do is only half of the problem of Friendliness; the other half is solving the problems in decision theory that will allow us to prove that an AI’s goal system and decision algorithms will cause it to not change its goal system and decision algorithms.
My understanding is that Eliezer considers this second part to be a substantially easier problem.
My understanding is that Eliezer considers this second part to be a substantially easier problem.