A paperclip maximizer would care about the amount of real paperclips in existence. Telling it that “oh, we’re going to destroy a million simulated paperclips” shouldn’t affect its decisions.
Of course, it might be badly programmed and confuse real and simulated paperclips when evaluating its future decisions, but one can’t rely on that. (It might also consider simulated paperclips to be just as real as physical ones, assuming the simulation met certain criteria, which isn’t obviously wrong. But again, can’t rely on that.)
A paperclip maximizer would care about the amount of real paperclips in existence. Telling it that “oh, we’re going to destroy a million simulated paperclips” shouldn’t affect its decisions.
Of course, it might be badly programmed and confuse real and simulated paperclips when evaluating its future decisions, but one can’t rely on that. (It might also consider simulated paperclips to be just as real as physical ones, assuming the simulation met certain criteria, which isn’t obviously wrong. But again, can’t rely on that.)