Now that would be a bummer—after all this sound and fury, not even a takeover of the galaxy?
Locally responding to this point, without commenting on the rest: I don’t think a paperclipped universe has much more value than an un-optimized universe (and it might even have less, e.g., because it results in a more homogeneous universe).
Also, aliens might exist somewhere in the observable universe; or they might come into existence in the future, if we don’t replace all the planets with paperclips. I’d expect the preferences of a random alien to be better than the preferences of a random human-built unaligned AGI, and having the universe get eaten by a paperclipper could destroy those other potential sources of value too.
Locally responding to this point, without commenting on the rest: I don’t think a paperclipped universe has much more value than an un-optimized universe (and it might even have less, e.g., because it results in a more homogeneous universe).
Also, aliens might exist somewhere in the observable universe; or they might come into existence in the future, if we don’t replace all the planets with paperclips. I’d expect the preferences of a random alien to be better than the preferences of a random human-built unaligned AGI, and having the universe get eaten by a paperclipper could destroy those other potential sources of value too.