Epistemic status: I am not an expert in the subject matter, so not confident, seems useful to have, I could not find anything discussion online.
As I understand it, Pascal Mugging can either be an honest opportunity for a large gain in utility (e.g. a Powerball ticket?) if the mugger is honest, or an adversarial attempt to exploit the agent (e.g. a Nigerian Prince Advance Fee Scam) if not. Most people have some skills in telling the two apart, usually erring on the side of caution. However, it is not clear how to formalize this approach. Most of the discussion focuses on ignoring super-tiny probabilities, and accepting a utility loss penalty, where a proverbial $10 on a busy sidewalk gets ignored because if it were real someone would have picked it up already, a situation humans navigate quite successfully most of the time.
I suspect that Pascal Mugging detection might be better approached from a different direction: instead of starting from “I know a liar when I see one” and the rule of thumb “Most too-good-to-be-true proposals are just that” and trying to formalize it for the case of “tiny probabilities of vast utilities”, one could try to start with a more general logic of “adversarial detection”, and then apply it to the special case of Pascal Mugging.
How would one go about telling apart an indifferent interaction, where something like the Solomonoff’s universal prior is appropriate, from an adversarial one, where it runs into trouble? Whatever the approach, it is likely to have the following properties:
In a non-adversarial case it reduces to the universal prior.
An attack by a “less-intelligent” adversary is severely penalized, the larger the difference in “intelligence” the harsher.
Conversely, an attack by a “vastly more intelligent” adversary is indistinguishable from the non-adversarial case.
In the Pascal Mugger’s case, the really clever one will get your money without you realizing what happened (see point 3 above), the dumb one will go away empty handed, and there is a whole range in between. There are probably some other desiderata that I am missing here.
Desiderata for an Adversarial Prior
Based on the discussion in the comments on https://www.lesswrong.com/posts/6XsZi9aFWdds8bWsy/is-there-any-discussion-on-avoiding-being-dutch-booked-or.
Epistemic status: I am not an expert in the subject matter, so not confident, seems useful to have, I could not find anything discussion online.
As I understand it, Pascal Mugging can either be an honest opportunity for a large gain in utility (e.g. a Powerball ticket?) if the mugger is honest, or an adversarial attempt to exploit the agent (e.g. a Nigerian Prince Advance Fee Scam) if not. Most people have some skills in telling the two apart, usually erring on the side of caution. However, it is not clear how to formalize this approach. Most of the discussion focuses on ignoring super-tiny probabilities, and accepting a utility loss penalty, where a proverbial $10 on a busy sidewalk gets ignored because if it were real someone would have picked it up already, a situation humans navigate quite successfully most of the time.
I suspect that Pascal Mugging detection might be better approached from a different direction: instead of starting from “I know a liar when I see one” and the rule of thumb “Most too-good-to-be-true proposals are just that” and trying to formalize it for the case of “tiny probabilities of vast utilities”, one could try to start with a more general logic of “adversarial detection”, and then apply it to the special case of Pascal Mugging.
How would one go about telling apart an indifferent interaction, where something like the Solomonoff’s universal prior is appropriate, from an adversarial one, where it runs into trouble? Whatever the approach, it is likely to have the following properties:
In a non-adversarial case it reduces to the universal prior.
An attack by a “less-intelligent” adversary is severely penalized, the larger the difference in “intelligence” the harsher.
Conversely, an attack by a “vastly more intelligent” adversary is indistinguishable from the non-adversarial case.
In the Pascal Mugger’s case, the really clever one will get your money without you realizing what happened (see point 3 above), the dumb one will go away empty handed, and there is a whole range in between. There are probably some other desiderata that I am missing here.