Choose A, and all life and sapience in the solar system (and presumably the universe), save for a sapient paperclipping AI, dies.
Choose B, and all life and sapience in the solar system, including the paperclipping AI, dies.
I choose A. (OTOH, the difference between U(A) and U(B) is so small that throwing even a small probability of a different C in the mix could easily change that.)
If anyone responds positively, subsequent questions would be which would be preferred, a paperclipper or a single bacteria; a paperclipper or a self-sustaining population of trilobites and their supporting ecology; a paperclipper or a self-sustaining population of australopithecines; and so forth, until the equivalent value is determined.
I’d take the paperclipper over the bacteria. I’d probably take the paperclipper over the trilobites and the australopithecines over the paperclipper, but I’m not very confident about that.
I’d take the paperclipper over the bacteria. I’d probably take the paperclipper over the trilobites [...]
I’m curious about your reasoning here. As others pointed out, a paperclipper is expected to be very stable, in the sense that it is plausible it will paperclip everything forever. Bacteria however have the potential to evolve a new ecosystem, and thus to lead to “people” existing again. (Admittedly, a single bacteria would need a very favorable environment.) And a paperclipper might even destroy/prevent life that would have evolved even without any bacteria at all. (After all, it happened at least once that we know of, and forever is a long time.)
I was more going with my gut feelings than with reasoning; anyway, thinking about the possibility of intelligent life arising again sounds like fighting the hypothetical to me (akin to thinking about the possibility of being incarcerated in the trolley dilemma), and also I’m not sure that there’s any guarantee that such a new intelligent life would be any more humane than the paperclipper.
Well, he did say “solar system (and presumably the universe)”. So considering the universe is stipulated in the hypothetical, but the “presumably” suggests the hypothetical does not dictate the universe. And given that the universe is much bigger than the solar system, it makes sense to me to think about it. (And hey, it’s hard to be less human than a paperclipper and still be intelligent. I thought that’s why we use paperclippers in these things.)
If the trolley problem mentioned “everybody on Earth” somewhere, it would be reasonable to actually consider other people than those on the track. Lesson: If you’re making a thought experiment about spherical cows in a vacuum, don’t mention pastures.
I choose A. (OTOH, the difference between U(A) and U(B) is so small that throwing even a small probability of a different C in the mix could easily change that.)
I’d take the paperclipper over the bacteria. I’d probably take the paperclipper over the trilobites and the australopithecines over the paperclipper, but I’m not very confident about that.
I’m curious about your reasoning here. As others pointed out, a paperclipper is expected to be very stable, in the sense that it is plausible it will paperclip everything forever. Bacteria however have the potential to evolve a new ecosystem, and thus to lead to “people” existing again. (Admittedly, a single bacteria would need a very favorable environment.) And a paperclipper might even destroy/prevent life that would have evolved even without any bacteria at all. (After all, it happened at least once that we know of, and forever is a long time.)
I was more going with my gut feelings than with reasoning; anyway, thinking about the possibility of intelligent life arising again sounds like fighting the hypothetical to me (akin to thinking about the possibility of being incarcerated in the trolley dilemma), and also I’m not sure that there’s any guarantee that such a new intelligent life would be any more humane than the paperclipper.
Well, he did say “solar system (and presumably the universe)”. So considering the universe is stipulated in the hypothetical, but the “presumably” suggests the hypothetical does not dictate the universe. And given that the universe is much bigger than the solar system, it makes sense to me to think about it. (And hey, it’s hard to be less human than a paperclipper and still be intelligent. I thought that’s why we use paperclippers in these things.)
If the trolley problem mentioned “everybody on Earth” somewhere, it would be reasonable to actually consider other people than those on the track. Lesson: If you’re making a thought experiment about spherical cows in a vacuum, don’t mention pastures.