Building one paperclip could EASILY increase the median and average number of future paperclips more than investing one paperclip’s worth of power into comet diversion.
Why do you think so? There will be no paperclips if planet and maximizer are destroyed.
There will be no paperclips if planet and maximizer are destroyed.
There might be—some paperclips could survive a comet. More importantly, one paperclip’s worth of resources won’t change the chance of a comet collision by any measurable amount, so the choice is either “completely waste that energy” or “make a paperclip that might survive”.
I don’t think your reasoning is mathematical. Worth of survival is infinite. And we have situation analogous to Pascal’s wager. Why do you think the maximizer would reject Pascal’s logic?
First rule of probability and decision theory: no infinities! If you want to postulate very large numbers, go ahead, but be prepared to deal with very tiny probabilities.
Pascal’s wager is a good example—the chance that the wager actually pays off based on this decision is infinitesimal (not zero, but small enough that I can’t really calculate with it), which makes it irrelevant how valuable it is. This gets even easier with the multitude of contradictory wagers on offer—“infinite value” from many different choices, only one of which can you take. Mostly, take the one(s) with lower value but actually believable conditional probability.
Why do you think it is rational to ignore tiny probabilities? I don’t think you can make maximizer ignore tiny probabilities. And some probabilities are not tiny, they are unknown (black swans), why do you think it is rational to ignore them?
In my opinion ignoring self preservation is contradictory to maximizer’s goal.
I understand that this is popular opinion, but it is not proven in any way. The opposite (focus on self preservation instead of paperclips) has logical proof (Pascal’s wager).
Why do you think so? There will be no paperclips if planet and maximizer are destroyed.
There might be—some paperclips could survive a comet. More importantly, one paperclip’s worth of resources won’t change the chance of a comet collision by any measurable amount, so the choice is either “completely waste that energy” or “make a paperclip that might survive”.
I don’t think your reasoning is mathematical. Worth of survival is infinite. And we have situation analogous to Pascal’s wager. Why do you think the maximizer would reject Pascal’s logic?
First rule of probability and decision theory: no infinities! If you want to postulate very large numbers, go ahead, but be prepared to deal with very tiny probabilities.
Pascal’s wager is a good example—the chance that the wager actually pays off based on this decision is infinitesimal (not zero, but small enough that I can’t really calculate with it), which makes it irrelevant how valuable it is. This gets even easier with the multitude of contradictory wagers on offer—“infinite value” from many different choices, only one of which can you take. Mostly, take the one(s) with lower value but actually believable conditional probability.
Why do you think it is rational to ignore tiny probabilities? I don’t think you can make maximizer ignore tiny probabilities. And some probabilities are not tiny, they are unknown (black swans), why do you think it is rational to ignore them? In my opinion ignoring self preservation is contradictory to maximizer’s goal. I understand that this is popular opinion, but it is not proven in any way. The opposite (focus on self preservation instead of paperclips) has logical proof (Pascal’s wager).
Maximizer can use robust decision making (https://en.wikipedia.org/wiki/Robust_decision-making) to deal with many contradictory choices.