As when we first discovered probability distributions in childhood, we may need to pay attention to medians, modes, variances, skewness, kurtosis or the overall shapes of the distributions. Pascal’s mugger and his whole family can be confronted head-on rather than hoping the probabilities neatly cancel out.
I would love to see the mugger dispelled, but part of the attraction of standard utility theory is that it seems very clean and optimal; is there any replacement axiom which convincingly deals with the low probability pathologies? Just going on current human psychology doesn’t seem very good.
One source of “the problem” seems to be a disguised version of unbounded payoffs.
Mugger: I can give you any finite amount of utility.
Victim: I find that highly unlikely.
Mugger: How unlikely?
Victim: 1/(really big number)
Mugger: Well, if you give me $1, I’ll give you (really big number)^2 times the utility of one dollar. Then your expected utility is positive, so you should give me the money.
The problem here is that whatever probability you give, the Mugger can always just make a better promise. Trying to assign “I can give you any finite amount of utility” a fixed non-zero probability is equivalent to assigning “I can give you an infinite amount of utility” a fixed non-zero probability. It’s sneaking an infinity in through the back door, so to speak.
It’s also very hard for any decision theory to deal with the problem “Name any rational number, and you get that much utility.” That’s because there is no largest rational number; no matter what number you name, there is another number that it is better to name. We can even come up with a version that even someone with a bounded utility function can be stumped by; “Name any rational number less than ten, and you get that much utility.” 9.9 is dominated by 9.99, which is dominated by 9.999, and so on. As long as you’re being asked to choose from a set that doesn’t contain its least upper bound, every choice is strictly dominated by some other choice. Even if all the numbers involved are finite, being given an infinite number of options can be enough to give decision theories the fits.
It’s sneaking an infinity in through the back door, so to speak.
Yes, this is precisely my own thinking—in order to give any assessment of the probability of the mugger delivering on any deal, you are in effect giving an assessment on an infinite number of deals (from 0 to infinity), and if you assign a non-zero probability to all of them (no matter how low), then you wind up with nonsensical results.
Giving the probability beforehand looks even worse if you ignore the deal aspect and simply ask what is the probability that anything the mugger says would be true? (Since this includes as a subset any promises to deliver utils.) Since he could make statements about turing machines or Chaitin’s Omega etc., now you’re into areas of intractable or undecidable questions!
As it happens, 2 or 3 days ago I emailed Bostrom about this. There was a followup paper to Bostrom’s “Pascal’s Mugging”, also published in Analysis, by a Baumann, who likewise rejected the prior probability, but Baumann didn’t have a good argument against it but to say that any such probability is ‘implausible’. Showing how infinities and undecidability get smuggled into the mugging shores up Baumann’s dismissal.
But once we’ve dismissed the prior probability, we still need to do something once the mugger has made a specific offer. If our probability doesn’t shrink at least as quickly as his offer increases, then we can still be mugged; if it shrinks exactly as quickly or even more quickly, we need to justify our specific shrinkage rate. And that is the perplexity: how fast do we shrink, and why?
(We want the Right theory & justification, not just one that is modeled after fallible humans or ad hocly makes the mugger go away. That is what I am asking for in the toplevel comment.)
Interesting thoughts on the mugger. But you still need a theory able to deal with it, not just an understanding of the problems.
For the second part, you can get a good decision theory for the “Name any rational number less than ten, and you get that much utility,” by giving you a certain fraction of negutility for each digit of your definition; there comes a time when the time wasted adding extra ’9’s dwarfs the gain in utility. See Tolstoy’s story How Much Land Does a Man Need for a traditional literary take on this problem.
The “Name any rational number, and you get that much utility” problem is more tricky, and would be a version of the “it is rational to spend infinity in hell” problem. Basically if your action (staying in hell; or specifying your utility) give you more ultimate utility than you lose by doing so, you will spend eternity doing your utility-losing action, and never cash in on your gained utility.
Replace expected utility by expected utility minus some multiple of the standard deviation, making that “some multiple” go to zero for oft repeated situations.
The mugger won’t be able to stand against that, as the standard deviation of his setup is huge.
Then you would turn down free money. Suppose you try to maximize EU—k*SD.
I’ll pick p < 1⁄2 * min(1, k^2), and offer you a bet in which you can receive 1 util with probability p, or 0 utils with probability (1-p). This bet has mean payout p and standard deviation sqrt[p(1-p)] You have nothing to lose, but you would turn down this bet.
Proof:
p < 1⁄2, so (1-p) > 1⁄2, so p < k^2/2 < k^2(1-p)
Divide both sides by (1-p): p / (1-p) < k^2
Take the square root of both sides: sqrt[p / (1-p)] < k
Multiply both sides by sqrt[p(1-p)]: p < k*sqrt[p(1-p)]
If k is tiny, this is only a minute chance of free money. I agree that it seems absurd to turn down that deal, but if the only cost of solving Pascal’s mugger is that we avoid advantageous lotteries with such minute payoffs, it seems a cost worth paying.
But recall—k is not a constant, it is a function of how often the “situation” is repeated. In this context, “repeated situation” means another lottery with larger standard deviation. I’d guess I’ve faced over a million implicit lotteries with SD higher than k = 0.1 in my life so far.
We can even get more subtle about the counting. For any SD we have faced that is n times greater than the SD of this lottery, we add n to 1/k.
In that setup, it may be impossible for you to actually propose that free money deal to me (I’ll have to check the maths—it certainly is impossible if we add n^3 to 1/k). Basically, the problem is that k depends on the SD, and the SD depends on k. As you diminish the SD to catch up with k, you further decrease k, and hence p, and hence the SD, and hence k, etc...
Interesting example, though; and I’ll try and actually formalise an example of a sensible “SD adjusted EU” so we can have proper debates about it.
That seems pretty arbitrary. You can make the mugging go away by simply penalizing his promise of n utils with a probability of 1/n (or less); but just making him go away is not a justification for such a procedure—what if you live in a universe where a eccentric god will give you that many utilons if you win his cosmic lottery?
I would love to see the mugger dispelled, but part of the attraction of standard utility theory is that it seems very clean and optimal; is there any replacement axiom which convincingly deals with the low probability pathologies? Just going on current human psychology doesn’t seem very good.
A thought on Pascal’s Mugging:
One source of “the problem” seems to be a disguised version of unbounded payoffs.
Mugger: I can give you any finite amount of utility.
Victim: I find that highly unlikely.
Mugger: How unlikely?
Victim: 1/(really big number)
Mugger: Well, if you give me $1, I’ll give you (really big number)^2 times the utility of one dollar. Then your expected utility is positive, so you should give me the money.
The problem here is that whatever probability you give, the Mugger can always just make a better promise. Trying to assign “I can give you any finite amount of utility” a fixed non-zero probability is equivalent to assigning “I can give you an infinite amount of utility” a fixed non-zero probability. It’s sneaking an infinity in through the back door, so to speak.
It’s also very hard for any decision theory to deal with the problem “Name any rational number, and you get that much utility.” That’s because there is no largest rational number; no matter what number you name, there is another number that it is better to name. We can even come up with a version that even someone with a bounded utility function can be stumped by; “Name any rational number less than ten, and you get that much utility.” 9.9 is dominated by 9.99, which is dominated by 9.999, and so on. As long as you’re being asked to choose from a set that doesn’t contain its least upper bound, every choice is strictly dominated by some other choice. Even if all the numbers involved are finite, being given an infinite number of options can be enough to give decision theories the fits.
Yes, this is precisely my own thinking—in order to give any assessment of the probability of the mugger delivering on any deal, you are in effect giving an assessment on an infinite number of deals (from 0 to infinity), and if you assign a non-zero probability to all of them (no matter how low), then you wind up with nonsensical results.
Giving the probability beforehand looks even worse if you ignore the deal aspect and simply ask what is the probability that anything the mugger says would be true? (Since this includes as a subset any promises to deliver utils.) Since he could make statements about turing machines or Chaitin’s Omega etc., now you’re into areas of intractable or undecidable questions!
As it happens, 2 or 3 days ago I emailed Bostrom about this. There was a followup paper to Bostrom’s “Pascal’s Mugging”, also published in Analysis, by a Baumann, who likewise rejected the prior probability, but Baumann didn’t have a good argument against it but to say that any such probability is ‘implausible’. Showing how infinities and undecidability get smuggled into the mugging shores up Baumann’s dismissal.
But once we’ve dismissed the prior probability, we still need to do something once the mugger has made a specific offer. If our probability doesn’t shrink at least as quickly as his offer increases, then we can still be mugged; if it shrinks exactly as quickly or even more quickly, we need to justify our specific shrinkage rate. And that is the perplexity: how fast do we shrink, and why?
(We want the Right theory & justification, not just one that is modeled after fallible humans or ad hocly makes the mugger go away. That is what I am asking for in the toplevel comment.)
Interesting thoughts on the mugger. But you still need a theory able to deal with it, not just an understanding of the problems.
For the second part, you can get a good decision theory for the “Name any rational number less than ten, and you get that much utility,” by giving you a certain fraction of negutility for each digit of your definition; there comes a time when the time wasted adding extra ’9’s dwarfs the gain in utility. See Tolstoy’s story How Much Land Does a Man Need for a traditional literary take on this problem.
The “Name any rational number, and you get that much utility” problem is more tricky, and would be a version of the “it is rational to spend infinity in hell” problem. Basically if your action (staying in hell; or specifying your utility) give you more ultimate utility than you lose by doing so, you will spend eternity doing your utility-losing action, and never cash in on your gained utility.
All I want for Christmas is an arbitrarily large chunk of utility.
Do you maybe see a problem with this concept?
Replace expected utility by expected utility minus some multiple of the standard deviation, making that “some multiple” go to zero for oft repeated situations.
The mugger won’t be able to stand against that, as the standard deviation of his setup is huge.
Then you would turn down free money. Suppose you try to maximize EU—k*SD.
I’ll pick p < 1⁄2 * min(1, k^2), and offer you a bet in which you can receive 1 util with probability p, or 0 utils with probability (1-p). This bet has mean payout p and standard deviation sqrt[p(1-p)] You have nothing to lose, but you would turn down this bet.
Proof:
p < 1⁄2, so (1-p) > 1⁄2, so p < k^2/2 < k^2(1-p)
Divide both sides by (1-p): p / (1-p) < k^2
Take the square root of both sides: sqrt[p / (1-p)] < k
Multiply both sides by sqrt[p(1-p)]: p < k*sqrt[p(1-p)]
Which is equivalent to: EU < k * SD
So EU—k*SD < 0
If k is tiny, this is only a minute chance of free money. I agree that it seems absurd to turn down that deal, but if the only cost of solving Pascal’s mugger is that we avoid advantageous lotteries with such minute payoffs, it seems a cost worth paying.
But recall—k is not a constant, it is a function of how often the “situation” is repeated. In this context, “repeated situation” means another lottery with larger standard deviation. I’d guess I’ve faced over a million implicit lotteries with SD higher than k = 0.1 in my life so far.
We can even get more subtle about the counting. For any SD we have faced that is n times greater than the SD of this lottery, we add n to 1/k.
In that setup, it may be impossible for you to actually propose that free money deal to me (I’ll have to check the maths—it certainly is impossible if we add n^3 to 1/k). Basically, the problem is that k depends on the SD, and the SD depends on k. As you diminish the SD to catch up with k, you further decrease k, and hence p, and hence the SD, and hence k, etc...
Interesting example, though; and I’ll try and actually formalise an example of a sensible “SD adjusted EU” so we can have proper debates about it.
That seems pretty arbitrary. You can make the mugging go away by simply penalizing his promise of n utils with a probability of 1/n (or less); but just making him go away is not a justification for such a procedure—what if you live in a universe where a eccentric god will give you that many utilons if you win his cosmic lottery?