you wouldn’t actually be willing [I don’t think?] to make such a trade.
Why shouldn’t I be? A 10^(-500) chance of utility 10^(750) yields an expected utility of 10^(250). This sounds like a pretty good deal to me, especially when you consider that “expected utility” is the technical term for “how good the deal is”.
(I’ll note at this point that we’re no longer discussing Pascal’s mugging, which is a problem in epistemology, about how we know the probability of the mugger’s threat is so low; instead, we’re discussing ordinary expected utility maximization.)
The mugger also doesn’t have to do all the work of raising your probability by a factor of 10^(500), the universe can do most (or all) of it. Remember, your priors are fixed once and for all at the beginning of time.
You postulated that my prior was 10^(-1000), and that the mugger raised it to 10^(-500). If other forces in the universe cooperated with the mugger to accomplish this, I don’t see how that changes the decision problem.
In the grand scheme of things, 10^(500) isn’t all that much. It’s just 1661 bits.
In which case, we can also say that a posterior probability of 10^(-500) is “just” 1661 bits away from even odds.
“expected utility” is the technical term for “how good the deal is”.
I know what the definition of utility is. My claim is that there does not exist any event such that you would care about it happening with probability 10^(-500) enough to pay $5.
You postulated that my prior was 10^(-1000), and that the mugger raised it to 10^(-500). If other forces in the universe cooperated with the mugger to accomplish this, I don’t see how that changes the decision problem.
You said that you would be okay with losing $5 to a mugger who raised your posterior by a factor of 10^(500), because they would have to do a lot of work to do so. I responded by pointing out that they wouldn’t have to do much work after all. If this doesn’t change the decision problem (which I agree with) then I don’t see how your original reasoning that it’s okay to get mugged because the mugger would have to work hard to mug you makes any sense.
At the very least, I consider making contradictory [and in the first case, rather flippant] responses to my comments to be somewhat logically rude, although I understand that you are the OP on this thread, and thus have to reply to many people’s comments and might not remember what you’ve said to me.
I believe that this entire back-and-forth is derailing the discussion, so I’m going to back up a few levels and try to start over.
In which case, we can also say that a posterior probability of 10^(-500) is “just” 1661 bits away from even odds.
You said that you would be okay with losing $5 to a mugger who raised your posterior by a factor of 10^(500), because they would have to do a lot of work to do so. I responded by pointing out that they wouldn’t have to do much work after all. If this doesn’t change the decision problem (which I agree with) then I don’t see how your original reasoning that it’s okay to get mugged because the mugger would have to work hard to mug you makes any sense.
What determines how much I am willing to pay is not how hard the mugger works per se, but how credible the threat is compared to its severity. (I thought this went without saying, and that you would be able to automatically generalize from “the mugger working hard” to “the mugger’s credibility increasing by whatever means”.) Going from p = 10^(-1000) to p = 10^(-500) may not sound like a “huge” increase in credibility, but it is. Or at least, if you insist that it isn’t, then you also have to concede that going from p = 10^(-500) to p = 1⁄2 isn’t that big of a credibility increase either, because it’s the same number of bits. In fact, measured in bits, going from p = 10^(-1000) to p = 10^(-500) is one-third of the way to p = 1-10^(-500) !
Now I presume you understand this arithmetic, so I agree that this is a distraction. In the same way, I think the simple mathematical arguments that you have been presenting are also a distraction. The real issue is that you apparently don’t believe that there exist outcomes with utilities in the range of 10^(750). Well, I am undecided on that question, because at this point I don’t know what “my” values look like in the limit of superintelligent extrapolation on galactic scales. (I like to think I’m pretty good at introspection, but I’m not that good!) But there’s no way I’m going to be convinced that my utility function has necessarily to be bounded without some serious argument going significantly beyond the fact that the consequences of an unbounded utility function seem counterintuitive to another human whose style of thought has already been demonstrated to be different from my own.
If you’ve got serious, novel arguments to offer for why a human-extracted utility function must be bounded, I’m quite willing to consider them, of course. But as of now I don’t have much evidence that you do have such arguments, because as far as I can tell, all you’ve said so far is “I can’t imagine anything with such high utility!”
P.S. Given that we’ve apparently had protracted disagreements on two issues so far, I just wanted you to know that I’m not trying to troll you or anything (in fact, I hadn’t realized that you were the same person who had made the Amanda Knox post). I will try to keep in mind in the future that our thinking styles are different and that appeals to intuition will probably just result in frustration.
Why shouldn’t I be? A 10^(-500) chance of utility 10^(750) yields an expected utility of 10^(250). This sounds like a pretty good deal to me, especially when you consider that “expected utility” is the technical term for “how good the deal is”.
(I’ll note at this point that we’re no longer discussing Pascal’s mugging, which is a problem in epistemology, about how we know the probability of the mugger’s threat is so low; instead, we’re discussing ordinary expected utility maximization.)
You postulated that my prior was 10^(-1000), and that the mugger raised it to 10^(-500). If other forces in the universe cooperated with the mugger to accomplish this, I don’t see how that changes the decision problem.
In which case, we can also say that a posterior probability of 10^(-500) is “just” 1661 bits away from even odds.
I know what the definition of utility is. My claim is that there does not exist any event such that you would care about it happening with probability 10^(-500) enough to pay $5.
You said that you would be okay with losing $5 to a mugger who raised your posterior by a factor of 10^(500), because they would have to do a lot of work to do so. I responded by pointing out that they wouldn’t have to do much work after all. If this doesn’t change the decision problem (which I agree with) then I don’t see how your original reasoning that it’s okay to get mugged because the mugger would have to work hard to mug you makes any sense.
At the very least, I consider making contradictory [and in the first case, rather flippant] responses to my comments to be somewhat logically rude, although I understand that you are the OP on this thread, and thus have to reply to many people’s comments and might not remember what you’ve said to me.
I believe that this entire back-and-forth is derailing the discussion, so I’m going to back up a few levels and try to start over.
Granted.
What determines how much I am willing to pay is not how hard the mugger works per se, but how credible the threat is compared to its severity. (I thought this went without saying, and that you would be able to automatically generalize from “the mugger working hard” to “the mugger’s credibility increasing by whatever means”.) Going from p = 10^(-1000) to p = 10^(-500) may not sound like a “huge” increase in credibility, but it is. Or at least, if you insist that it isn’t, then you also have to concede that going from p = 10^(-500) to p = 1⁄2 isn’t that big of a credibility increase either, because it’s the same number of bits. In fact, measured in bits, going from p = 10^(-1000) to p = 10^(-500) is one-third of the way to p = 1-10^(-500) !
Now I presume you understand this arithmetic, so I agree that this is a distraction. In the same way, I think the simple mathematical arguments that you have been presenting are also a distraction. The real issue is that you apparently don’t believe that there exist outcomes with utilities in the range of 10^(750). Well, I am undecided on that question, because at this point I don’t know what “my” values look like in the limit of superintelligent extrapolation on galactic scales. (I like to think I’m pretty good at introspection, but I’m not that good!) But there’s no way I’m going to be convinced that my utility function has necessarily to be bounded without some serious argument going significantly beyond the fact that the consequences of an unbounded utility function seem counterintuitive to another human whose style of thought has already been demonstrated to be different from my own.
If you’ve got serious, novel arguments to offer for why a human-extracted utility function must be bounded, I’m quite willing to consider them, of course. But as of now I don’t have much evidence that you do have such arguments, because as far as I can tell, all you’ve said so far is “I can’t imagine anything with such high utility!”
Fair enough.
P.S. Given that we’ve apparently had protracted disagreements on two issues so far, I just wanted you to know that I’m not trying to troll you or anything (in fact, I hadn’t realized that you were the same person who had made the Amanda Knox post). I will try to keep in mind in the future that our thinking styles are different and that appeals to intuition will probably just result in frustration.