There’s also the issue that real AIs, unlike the imaginary AIs, have to be made of parts, which are made of parts, down to unintelligent parts. A paperclip maximizer would need to subdivide effort—let one part think of copper paperclips, other of iron paperclips, and the third, of stabilized metallic hydrogen paperclips. And they shouldn’t hack each other or think of hacking each other or the like.
There’s also the issue that real AIs, unlike the imaginary AIs, have to be made of parts, which are made of parts, down to unintelligent parts. A paperclip maximizer would need to subdivide effort—let one part think of copper paperclips, other of iron paperclips, and the third, of stabilized metallic hydrogen paperclips. And they shouldn’t hack each other or think of hacking each other or the like.