only this time when we update on the mugger telling the truth, we radically change our estimate of the number of people who were ‘in the lottery’, all the way up to 3^^^^3. We then multiply 1/3^^^^3 by the probability that we live in a universe where Pascal’s muggings occur
How does this work with Clippy (the only paperclipper in known existence) being tempted with 3^^^^3 paperclips?
That’s part of why I dislike Robin Hanson’s original solution. That the tempting/blackmailing offer involves 3^^^^3 other people, and that you are also a person should be merely incidental to one particular illustration of the problem of Pascal’s Mugging—and as such it can’t be part of a solution to the core problem.
To replace this with something like “causal nodes”, as Eliezer mentions, might perhaps solve the problem. But I wish that we started talking about Clippy and his paperclips instead, so that the original illustration of the problem which involves incidental symmetries doesn’t mislead us into a “solution” overreliant on symmetries.
How does this work with Clippy (the only paperclipper in known existence) being tempted with 3^^^^3 paperclips?
Clippy has some sort of prior over the number of paperclips that could possibly exist. Let this number be P. Conditioned on each value of P, Clippy evaluates the utility of the offer and the probability that it comes true.
In particular, for P < 3^^^^3, the conditional probability that the offer of 3^^^^3 paperclips is legit is 0. If some large number of paperclips exists, e.g. P = 2*3^^^^3, the offer might actually be viable with non-negligible probability, while its utility would be given by 3^^^^3/P. Note that this is always at most 1.
However, unless Clippy lives in a very strange universe, it thinks that P >= 3^^^^3 is very unlikely. So the expected utility will be bounded by Pr[P >= 3^^^^3] and will end up being very small.
How does this work with Clippy (the only paperclipper in known existence) being tempted with 3^^^^3 paperclips?
First thought, I’m not at all sure that it does. Pascal’s mugging may still be a problem. This doesn’t seem to contradict what I said about the leverage penalty being the only correct approach, rather than a ‘fix’ of some kind, in the first case. Worryingly, if you are correct it may also not be a ‘fix’ in the sense of not actually fixing anything.
I notice I’m currently confused about whether the ‘causal nodes’ patch is justified by the same argument. I will think about it and hopefully find an answer.
How does this work with Clippy (the only paperclipper in known existence) being tempted with 3^^^^3 paperclips?
That’s part of why I dislike Robin Hanson’s original solution. That the tempting/blackmailing offer involves 3^^^^3 other people, and that you are also a person should be merely incidental to one particular illustration of the problem of Pascal’s Mugging—and as such it can’t be part of a solution to the core problem.
To replace this with something like “causal nodes”, as Eliezer mentions, might perhaps solve the problem. But I wish that we started talking about Clippy and his paperclips instead, so that the original illustration of the problem which involves incidental symmetries doesn’t mislead us into a “solution” overreliant on symmetries.
Clippy has some sort of prior over the number of paperclips that could possibly exist. Let this number be P. Conditioned on each value of P, Clippy evaluates the utility of the offer and the probability that it comes true.
In particular, for P < 3^^^^3, the conditional probability that the offer of 3^^^^3 paperclips is legit is 0. If some large number of paperclips exists, e.g. P = 2*3^^^^3, the offer might actually be viable with non-negligible probability, while its utility would be given by 3^^^^3/P. Note that this is always at most 1.
However, unless Clippy lives in a very strange universe, it thinks that P >= 3^^^^3 is very unlikely. So the expected utility will be bounded by Pr[P >= 3^^^^3] and will end up being very small.
First thought, I’m not at all sure that it does. Pascal’s mugging may still be a problem. This doesn’t seem to contradict what I said about the leverage penalty being the only correct approach, rather than a ‘fix’ of some kind, in the first case. Worryingly, if you are correct it may also not be a ‘fix’ in the sense of not actually fixing anything.
I notice I’m currently confused about whether the ‘causal nodes’ patch is justified by the same argument. I will think about it and hopefully find an answer.
This sounds a little bit like it might depend on the choice of SSA vs. SIA.