You can skip this pararaph and the next if you’re familiar with the problem. But if you’re not, here’s an illustration. Suppose your friend has some pennies that she would like to arrange into a rectangle, which of course is impossible if the number of pennies is prime. Let’s call the number of pennies N. Your friend would like to use probability theory to guess whether it’s worth trying; if there’s a 50% chance that Prime(N), she won’t bother trying to make the rectangle. You might imagine that if she counts them and finds that there’s an odd number, this is evidence of Prime(N); if she furthermore notices that the digits don’t sum to a multiple of three, this is further evidence of Prime(N). In general, each test of compositeness that she knows should, if it fails, raise the probability of Prime(N).
But what happens instead is this. Suppose you both count them, and find that N=53. Being a LessWrong reader, you of course recognize from recently posted articles that N=53 implies Prime(N), though she does not. But this means that P(N=53) ⇐ P(Prime(N)). If you’re quite sure of N=53—that is, P(N=53) is near 1—then P(Prime(N)) is also near 1. There’s no way for her to get a gradient of uncertainty from simple tests of compositeness. The probability is just some number near 1.
I don’t understand why this is a problem. You and your friend have different states of knowledge, so you assign different probabilities.
One of his “desiderata”, his principles of construction, is that the robot gives equal plausibility assignments to logically equivalent statements
I don’t see this desiderata. The consistency requirement is that if there are multiple ways of calculating something, then all of them yield the same result. A few minutes of thought didn’t lead to any way of leveraging a non 1 or zero probability for Prime(53) into two different results.
If I try to do anything with P(Prime(53)|PA), I get stuff like P(PA|Prime(53)), and I don’t have any idea how to interpret that. Since PA is a set of axioms, it doesn’t really have a truth value that we can do probability with. Technically speaking, Prime(N) means that the PA axioms imply that 53 has two factors. Since the axioms are in the predicate, any mechanism that forces P(Prime(53)) to be one must do so for all priors.
One final thing: Isn’t it wrong to assign a probability of zero to Prime(4), i.e. PA implies that 4 has two factors, since PA could be inconsistent and imply everything?
I now think you’re right that logical uncertainty doesn’t violate any of Jaynes’s desiderata. Which means I should probably try to follow them more closely, if they don’t create problems like I thought they would.
I don’t understand why this is a problem. You and your friend have different states of knowledge, so you assign different probabilities.
the problem is that, in probability theory as usually formalized and discussed, we assign the same probabilities, though we shouldn’t… do you see?
EDIT: and it’s a problem because she can’t calculate her probability without proving whether or not it’s prime.
I don’t see this desiderata. The consistency requirement is that if there are multiple ways of calculating something, then all of them yield the same result. A few minutes of thought didn’t lead to any way of leveraging a non 1 or zero probability for Prime(53) into two different results.
If I try to do anything with P(Prime(53)|PA), I get stuff like P(PA|Prime(53)), and I don’t have any idea how to interpret that. Since PA is a set of axioms, it doesn’t really have a truth value that we can do probability with. Technically speaking, Prime(N) means that the PA axioms imply that 53 has two factors. Since the axioms are in the predicate, any mechanism that forces P(Prime(53)) to be one must do so for all priors.
One final thing: Isn’t it wrong to assign a probability of zero to Prime(4), i.e. PA implies that 4 has two factors, since PA could be inconsistent and imply everything?
I now think you’re right that logical uncertainty doesn’t violate any of Jaynes’s desiderata. Which means I should probably try to follow them more closely, if they don’t create problems like I thought they would.
An Aspiring Rationalist’s Ramble has a post asserting the same thing, that nothing in the desiderata implies logical omniscience.