That sounds like the paperclipper is getting Pascal’s Mugged by its own reasoning. Sure, it’s possible that there’s a minor action (such as not sending me $5 via Paypal) that leads to a whole bunch of paperclips being destroyed; but the probability of that is low, and the paperclipper ought to focus on more high-probability paperclipping plans instead.
Well, that depends to choice of prior. Some priors don’t penalize theories for the “size” of the hypothetical world, and in those, max. size of the world grows faster than any computable function of length if it’s description, and when you assign improbability depending to length of description, basically, it fails. Bigger issue is defining what the ‘real world paperclip count’ even is.
That sounds like the paperclipper is getting Pascal’s Mugged by its own reasoning. Sure, it’s possible that there’s a minor action (such as not sending me $5 via Paypal) that leads to a whole bunch of paperclips being destroyed; but the probability of that is low, and the paperclipper ought to focus on more high-probability paperclipping plans instead.
Well, that depends to choice of prior. Some priors don’t penalize theories for the “size” of the hypothetical world, and in those, max. size of the world grows faster than any computable function of length if it’s description, and when you assign improbability depending to length of description, basically, it fails. Bigger issue is defining what the ‘real world paperclip count’ even is.