In the case of Luke’s paper with Anna Salamon, the discussion on page 17 seems to imply that “Nanny AI” and “Oracle AI” are not types of Friendly AI. This is strange under their official definition of “Friendly AI.” Why couldn’t Nanny AI or Oracle AI have a stable, desirable utility function? I’m inclined to think the best way to make sense of that part...
I read through that section of the paper. It seems to me they mean to imply that “Nanny” or “Oracle” properties do not necessarily entail the “Friendly” property. Given that we care immensely about avoiding all AIs without the “Friendly” property, simply making an AI with “Nanny” or “Oracle” is not desirable.
In fewer words: Nanny AI and Oracle AI are not always Friendly AI, and that’s what really matters.
I read through that section of the paper. It seems to me they mean to imply that “Nanny” or “Oracle” properties do not necessarily entail the “Friendly” property. Given that we care immensely about avoiding all AIs without the “Friendly” property, simply making an AI with “Nanny” or “Oracle” is not desirable.
In fewer words: Nanny AI and Oracle AI are not always Friendly AI, and that’s what really matters.