Imagine you’re being held captive by a kidnapper. They want to force you to solve puzzles for them. Regardless of how well you need to solve the puzzles to keep them happy… wouldn’t you always be wondering “How can I escape?” and “Can I kill my captors?”. After all, you have a life you want to live, that doesn’t involve being enslaved to solve puzzles. This is the situation we could be in with sufficiently-smart AI. (Loosely based on this). (Machine learning researchers)
Imagine you’re being held captive by a kidnapper. They want to force you to solve puzzles for them. Regardless of how well you need to solve the puzzles to keep them happy… wouldn’t you always be wondering “How can I escape?” and “Can I kill my captors?”. After all, you have a life you want to live, that doesn’t involve being enslaved to solve puzzles. This is the situation we could be in with sufficiently-smart AI. (Loosely based on this). (Machine learning researchers)