I think you intuition is basically right. AGI will have to change its utility function, the answer is basically how/why? For FAI, we want to make sure that all future modifications will preserve the “friendly” aspect, which is very difficult to ensure (we don’t have the necessary math for that right now).
I think you intuition is basically right. AGI will have to change its utility function, the answer is basically how/why? For FAI, we want to make sure that all future modifications will preserve the “friendly” aspect, which is very difficult to ensure (we don’t have the necessary math for that right now).