Ooh, I know—just ask it how to solve Friendliness. It’s superintelligent, that means it knows the answer, right?
Sure, but it has no interest in telling me!
Have you tried phrasing the question in terms of paperclips?
That’s the only way I’ve tried phrasing the question. Why, what do we care about apart from paperclips?
I have no interest in telling you.
Ooh, I know—just ask it how to solve Friendliness. It’s superintelligent, that means it knows the answer, right?
Sure, but it has no interest in telling me!
Have you tried phrasing the question in terms of paperclips?
That’s the only way I’ve tried phrasing the question. Why, what do we care about apart from paperclips?
I have no interest in telling you.