The problem with Pascal’s Mugging is that in a bounded rationality situation, you cannot possibly have a probability on the order of 1/(3^^^^3) - it’s incomprehensibly low. Thus, a naive Bayesian expected utility maximizer might jump at this offer every time it’s offered despite it never being true—allowing one to get it to do anything for you, instead of whatever it was supposed to be maximizing.
It occurs to me that this problem shouldn’t crop up—you could make the effective limits on calculated utility be the same as the effective limits on calculated probability (and MAX = 1/MIN) - thus VERY HIGH utility times VERY LOW probability = MAX*MIN = 1, which is probably lower than the expected utility for whatever it was doing in the first place.
I’ll have to go re-read the original discussion.
ETA: Oh right, the whole point was that there needs to be some justification for the symmetrically-low probability, because it is not obvious from the structure of the problem like in Pascal’s Wager. Duh.
The problem with Pascal’s Mugging is that in a bounded rationality situation, you cannot possibly have a probability on the order of 1/(3^^^^3) - it’s incomprehensibly low. Thus, a naive Bayesian expected utility maximizer might jump at this offer every time it’s offered despite it never being true—allowing one to get it to do anything for you, instead of whatever it was supposed to be maximizing.
It occurs to me that this problem shouldn’t crop up—you could make the effective limits on calculated utility be the same as the effective limits on calculated probability (and MAX = 1/MIN) - thus VERY HIGH utility times VERY LOW probability = MAX*MIN = 1, which is probably lower than the expected utility for whatever it was doing in the first place.
I’ll have to go re-read the original discussion.
ETA: Oh right, the whole point was that there needs to be some justification for the symmetrically-low probability, because it is not obvious from the structure of the problem like in Pascal’s Wager. Duh.