First rule of probability and decision theory: no infinities! If you want to postulate very large numbers, go ahead, but be prepared to deal with very tiny probabilities.
Pascal’s wager is a good example—the chance that the wager actually pays off based on this decision is infinitesimal (not zero, but small enough that I can’t really calculate with it), which makes it irrelevant how valuable it is. This gets even easier with the multitude of contradictory wagers on offer—“infinite value” from many different choices, only one of which can you take. Mostly, take the one(s) with lower value but actually believable conditional probability.
Why do you think it is rational to ignore tiny probabilities? I don’t think you can make maximizer ignore tiny probabilities. And some probabilities are not tiny, they are unknown (black swans), why do you think it is rational to ignore them?
In my opinion ignoring self preservation is contradictory to maximizer’s goal.
I understand that this is popular opinion, but it is not proven in any way. The opposite (focus on self preservation instead of paperclips) has logical proof (Pascal’s wager).
First rule of probability and decision theory: no infinities! If you want to postulate very large numbers, go ahead, but be prepared to deal with very tiny probabilities.
Pascal’s wager is a good example—the chance that the wager actually pays off based on this decision is infinitesimal (not zero, but small enough that I can’t really calculate with it), which makes it irrelevant how valuable it is. This gets even easier with the multitude of contradictory wagers on offer—“infinite value” from many different choices, only one of which can you take. Mostly, take the one(s) with lower value but actually believable conditional probability.
Why do you think it is rational to ignore tiny probabilities? I don’t think you can make maximizer ignore tiny probabilities. And some probabilities are not tiny, they are unknown (black swans), why do you think it is rational to ignore them? In my opinion ignoring self preservation is contradictory to maximizer’s goal. I understand that this is popular opinion, but it is not proven in any way. The opposite (focus on self preservation instead of paperclips) has logical proof (Pascal’s wager).
Maximizer can use robust decision making (https://en.wikipedia.org/wiki/Robust_decision-making) to deal with many contradictory choices.