3. Even if you don’t accept 1 and 2 above, there’s no reason to expect that the person is telling the truth. He might kill the people even if you give him the $5, or conversely he might not kill them even if you don’t give him the $5.
But if a Bayesian AI actually calculates these probabilities by assessing their Kolmogorov complexity—or any other technique you like, for that matter—without desiring that they come out exactly equal, can you rely on them coming out exactly equal? If not, an expected utility differential of 2 to the negative googolplex times 3^^^^3 still equals 3^^^^3, so whatever tiny probability differences exist will dominate all calculations based on what we think of as the “real world” (the mainline of probability with no wizards).
if you have the imagination to imagine X to be super-huge, you should be able to have the imagination to imagine p to be super-small
But we can’t just set the probability to anything we like. We have to calculate it, and Kolmogorov complexity, the standard accepted method, will not be anywhere near that super-small.
3. Even if you don’t accept 1 and 2 above, there’s no reason to expect that the person is telling the truth. He might kill the people even if you give him the $5, or conversely he might not kill them even if you don’t give him the $5.
But if a Bayesian AI actually calculates these probabilities by assessing their Kolmogorov complexity—or any other technique you like, for that matter—without desiring that they come out exactly equal, can you rely on them coming out exactly equal? If not, an expected utility differential of 2 to the negative googolplex times 3^^^^3 still equals 3^^^^3, so whatever tiny probability differences exist will dominate all calculations based on what we think of as the “real world” (the mainline of probability with no wizards).
if you have the imagination to imagine X to be super-huge, you should be able to have the imagination to imagine p to be super-small
But we can’t just set the probability to anything we like. We have to calculate it, and Kolmogorov complexity, the standard accepted method, will not be anywhere near that super-small.