Ah! I’ve got it! I think… At least on the probability side.
The problem is our intuition: the utility of human lives grows linearly with the population of humans, while the message size of the hypothesis needed to describe them grows roughly logarithmically. Since we think we climb down the exponential decline in prior probability slower than utility increases, Pascal’s Bargain sounds favorable.
This is wrong. A real Solomonoff Hypothesis, after all, does not merely say “There are 3^^^3 humans” if there really are 3 ^^^ 3 humans. It describes and predicts each single human, after all, in detail. It’s a hypothesis that aims to predict an entire universe from one small piece of fairy cake.
And when you have to make predictive descriptions of humans, the summed size of those descriptions will grow at least linearly in the number of purported humans. God help you if they start organizing themselves into complicated societies, which are more complex than a mere summed set of single persons. Now your utility from taking the Bargain grows linearly while your negative exponent on the probability of its being real declines linearly.
The question then becomes a simple matter of where your plausability versus utility tradeoff sits for one human life.
Or in other words, if Pascal’s Mugger catches you in a back ally, you should demand that he start describing the people he’s threatening one-by-one.
Or in other words, if Pascal’s Mugger catches you in a back ally, you should demand that he start describing the people he’s threatening one-by-one.
By this reasoning, if someone has their finger on a trigger for a nuclear bomb that will destroy a city of a million people, and says “give me $1 or I will pick a random number and destroy the city at a 1/1000000 chance”, you should refuse. After all, he cannot describe any of the people he is planning to kill, so this is equivalent to killing one person at a 1/1000000 chance, and you could probably exceed that chance of killing someone just by driving a couple of miles to the supermarket.
If it’s a million people possibly dying at a one-in-a-million chance, then in expected-death terms he’s charging me $1 not to kill one person. Since I believe human lives are worth more than $1, I should give him the dollar and make a “profit”.
Of course, the other issue here, and the reason we don’t make analogies between serious military threats and Pascal’s Mugging, is that in the nuclear bomb case, there is actual evidence on which to update my beliefs. For instance, the button he’s got his finger on is either real, or not. If I can see damn well that it’s a plastic toy from the local store, I’ve no reason to give him a dollar.
So in the case of Soviet Russia threatening you, you’ve got real evidence that they might deliberately nuke your cities with a probability much higher than one in a million. In the case of Pascal’s Mugger, you’ve got a 1⁄1,000,000 chance that the mugger is telling the truth at all, and all the other probability mass points at the mugger being a delusion of your badly-coded reasoning algorithms.
If it’s a million people possibly dying at a one-in-a-million chance, and I use the reasoning you used before, because the mugger can’t describe the people he’s threatening to kill, I shouldn’t treat that as any worse than a threat to kill one person at a one-in-a-million chance.
You misconstrue my position. I’m not saying, “Descriptions are magic!”. I’m saying: I prefer evidentialism to pure Bayesianism. Meaning: if the mugger can’t describe anything about the city under threat, that is evidence that he is lying.
Which misses the point of the scenario, since a real Pascal’s Mugging is not about a physical mugger who could ever be lying. It’s about having a flaw in your own reasoning system.
Ah! I’ve got it! I think… At least on the probability side.
The problem is our intuition: the utility of human lives grows linearly with the population of humans, while the message size of the hypothesis needed to describe them grows roughly logarithmically. Since we think we climb down the exponential decline in prior probability slower than utility increases, Pascal’s Bargain sounds favorable.
This is wrong. A real Solomonoff Hypothesis, after all, does not merely say “There are 3^^^3 humans” if there really are 3 ^^^ 3 humans. It describes and predicts each single human, after all, in detail. It’s a hypothesis that aims to predict an entire universe from one small piece of fairy cake.
And when you have to make predictive descriptions of humans, the summed size of those descriptions will grow at least linearly in the number of purported humans. God help you if they start organizing themselves into complicated societies, which are more complex than a mere summed set of single persons. Now your utility from taking the Bargain grows linearly while your negative exponent on the probability of its being real declines linearly.
The question then becomes a simple matter of where your plausability versus utility tradeoff sits for one human life.
Or in other words, if Pascal’s Mugger catches you in a back ally, you should demand that he start describing the people he’s threatening one-by-one.
By this reasoning, if someone has their finger on a trigger for a nuclear bomb that will destroy a city of a million people, and says “give me $1 or I will pick a random number and destroy the city at a 1/1000000 chance”, you should refuse. After all, he cannot describe any of the people he is planning to kill, so this is equivalent to killing one person at a 1/1000000 chance, and you could probably exceed that chance of killing someone just by driving a couple of miles to the supermarket.
If it’s a million people possibly dying at a one-in-a-million chance, then in expected-death terms he’s charging me $1 not to kill one person. Since I believe human lives are worth more than $1, I should give him the dollar and make a “profit”.
Of course, the other issue here, and the reason we don’t make analogies between serious military threats and Pascal’s Mugging, is that in the nuclear bomb case, there is actual evidence on which to update my beliefs. For instance, the button he’s got his finger on is either real, or not. If I can see damn well that it’s a plastic toy from the local store, I’ve no reason to give him a dollar.
So in the case of Soviet Russia threatening you, you’ve got real evidence that they might deliberately nuke your cities with a probability much higher than one in a million. In the case of Pascal’s Mugger, you’ve got a 1⁄1,000,000 chance that the mugger is telling the truth at all, and all the other probability mass points at the mugger being a delusion of your badly-coded reasoning algorithms.
If it’s a million people possibly dying at a one-in-a-million chance, and I use the reasoning you used before, because the mugger can’t describe the people he’s threatening to kill, I shouldn’t treat that as any worse than a threat to kill one person at a one-in-a-million chance.
You misconstrue my position. I’m not saying, “Descriptions are magic!”. I’m saying: I prefer evidentialism to pure Bayesianism. Meaning: if the mugger can’t describe anything about the city under threat, that is evidence that he is lying.
Which misses the point of the scenario, since a real Pascal’s Mugging is not about a physical mugger who could ever be lying. It’s about having a flaw in your own reasoning system.