Pascal’s Mugging has always confused me. It relies on the assumption that the likelihood of the payoff diminishes more slowly than the size of the payoff.
I can imagine regions of payoff vs. likelihood graphs where that’s true. But in general, I expect that likelihood diminishes with greater acceleration than the acceleration of payoff. So eventually, likelihood diminishes faster than payoff increases, even if that was not the case at first. This lets me avoid Pascal’s Muggings.
I can imagine an intelligence that somehow gets confused and misses this point. Or one that is smarter than us and behaves in a way that we would recognize as a Pascal’s Mugging, but which it might be able to justify and persuade us of.
Am I missing a point here? Or is Pascal’s Mugging just a description of one of the enormous number of ways it is possible to be dumb?
It’s a mistake to think that likelihood or payoff changes—each instance is independent. And the reason it’s effective is the less-explored alternate universe where you get PAID just for being the kind of person who’d accept it. In the setup, this is stated to be believed by you (meaning: you can’t just say “but it’s low probability!” that’s rejecting the premise). That said I agree with your description, with a fairly major modification. Instead of
Pascal’s Mugging just a description of one of the enormous number of ways it is possible to be dumb
I’d say Pascal’s Mugging is just a description of one of the enormous number of ways that a superior predictor/manipulator can set up human-level decision processes to fail. Adversarial situations against a better modeler are pretty much hopeless.
There are a number of ways to deal with this, however, the succinct realization is that Pascal’s mugging isn’t something you do to yourself. Another player is telling you the expected reward, and has made it arbitrarily large.
Therefore this assessment of future reward is potentially hostile misinformation. It’s been manipulated. For better or for worse, the way contemporary institutions typically deal with this problem is to simply assume any untrustworthy information is exactly zero probability. None at all. The issue with this comes up that contemporary institutions use “has a degree in the field/peer acclaim” as a way to identify who might have something trustworthy to say, and weight “has analyzed the raw data with correct math but is some random joe” as falling in that zero case.
This is where we end up with all kinds of failures and one of the many problems we need a form of AI to solve.
But yes you have hit on a way to filter potentially untrustworthy information without just throwing it out. In essence, you currently have a belief and confidence. Someone has some potentially untrustworthy information that differs from your belief. Your confidence in that data should decrease faster than the difference between the information and your present belief.
I agree with your analysis. For a Pascal’s mugging to work, you need to underestimate how small Pr(your interlocutor is willing and able to change your utility by x) gets when x gets large. Human beings are bad at estimating that, which is why when we explicitly consider PM-type situations in an expected-utility framework we may get the wrong answer; it’s possible that there is a connection between this and the fact (which generally saves us from PM-like problems in real life) that we tend to round very small probabilities to zero, whether explicitly or implicitly by simply dismissing someone who comes to us with PM-type claims.
Pascal’s Mugging has always confused me. It relies on the assumption that the likelihood of the payoff diminishes more slowly than the size of the payoff.
I can imagine regions of payoff vs. likelihood graphs where that’s true. But in general, I expect that likelihood diminishes with greater acceleration than the acceleration of payoff. So eventually, likelihood diminishes faster than payoff increases, even if that was not the case at first. This lets me avoid Pascal’s Muggings.
I can imagine an intelligence that somehow gets confused and misses this point. Or one that is smarter than us and behaves in a way that we would recognize as a Pascal’s Mugging, but which it might be able to justify and persuade us of.
Am I missing a point here? Or is Pascal’s Mugging just a description of one of the enormous number of ways it is possible to be dumb?
It’s a mistake to think that likelihood or payoff changes—each instance is independent. And the reason it’s effective is the less-explored alternate universe where you get PAID just for being the kind of person who’d accept it. In the setup, this is stated to be believed by you (meaning: you can’t just say “but it’s low probability!” that’s rejecting the premise). That said I agree with your description, with a fairly major modification. Instead of
I’d say Pascal’s Mugging is just a description of one of the enormous number of ways that a superior predictor/manipulator can set up human-level decision processes to fail. Adversarial situations against a better modeler are pretty much hopeless.
There are a number of ways to deal with this, however, the succinct realization is that Pascal’s mugging isn’t something you do to yourself. Another player is telling you the expected reward, and has made it arbitrarily large.
Therefore this assessment of future reward is potentially hostile misinformation. It’s been manipulated. For better or for worse, the way contemporary institutions typically deal with this problem is to simply assume any untrustworthy information is exactly zero probability. None at all. The issue with this comes up that contemporary institutions use “has a degree in the field/peer acclaim” as a way to identify who might have something trustworthy to say, and weight “has analyzed the raw data with correct math but is some random joe” as falling in that zero case.
This is where we end up with all kinds of failures and one of the many problems we need a form of AI to solve.
But yes you have hit on a way to filter potentially untrustworthy information without just throwing it out. In essence, you currently have a belief and confidence. Someone has some potentially untrustworthy information that differs from your belief. Your confidence in that data should decrease faster than the difference between the information and your present belief.
I agree with your analysis. For a Pascal’s mugging to work, you need to underestimate how small Pr(your interlocutor is willing and able to change your utility by x) gets when x gets large. Human beings are bad at estimating that, which is why when we explicitly consider PM-type situations in an expected-utility framework we may get the wrong answer; it’s possible that there is a connection between this and the fact (which generally saves us from PM-like problems in real life) that we tend to round very small probabilities to zero, whether explicitly or implicitly by simply dismissing someone who comes to us with PM-type claims.