I got that! The problem was more that I was thinking as if the world could be divided up into sealable boxes. In practice, we can do a lot focusing on one area with ‘no effect’ on anything else. But this is because the sorts of actions we do are limited, we can’t detect the low-level impact on things outside those boxes and we have certain unspoken understandings about what sort of thing might constitute an unacceptable effect elsewhere (if I only care about looking at pictures of LOLcats, I might be ‘neutral to the rest of the internet’ except for taking a little bandwidth. A superpowerful AI might realise that it would slightly increase the upload speed and thus maximise the utility function if vast swathes of other internet users were dead).
(if I only care about looking at pictures of LOLcats, I might be ‘neutral to the rest of the internet’ except for taking a little bandwidth. A superpowerful AI might realise that it would slightly increase the upload speed and thus maximise the utility function if vast swathes of other internet users were dead).
Dead and hilariously captioned, possibly.
I bet such an AI could latch onto a meme in which the absence of cats in such pictures was “lulz”.
I got that! The problem was more that I was thinking as if the world could be divided up into sealable boxes. In practice, we can do a lot focusing on one area with ‘no effect’ on anything else. But this is because the sorts of actions we do are limited, we can’t detect the low-level impact on things outside those boxes and we have certain unspoken understandings about what sort of thing might constitute an unacceptable effect elsewhere (if I only care about looking at pictures of LOLcats, I might be ‘neutral to the rest of the internet’ except for taking a little bandwidth. A superpowerful AI might realise that it would slightly increase the upload speed and thus maximise the utility function if vast swathes of other internet users were dead).
Dead and hilariously captioned, possibly.
I bet such an AI could latch onto a meme in which the absence of cats in such pictures was “lulz”.