I have a problem understanding why a utility function would ever “stick” to an AI, to actually become something that it wants to keep pursuing.
I think that’s one of MIRI’s research problems. Designing an self-modifying AI that doesn’t change it’s utility function isn’t trival.
I think that’s one of MIRI’s research problems. Designing an self-modifying AI that doesn’t change it’s utility function isn’t trival.