If you have all the relevant probabilities, you have all the information you need to calculate expected utilities for all possible choices—you don’t need to decide on how “metaconfident” you are in these probabilities. Your probabilities may not be based on very good data, and so you might anticipate that these probabilities will change drastically when you update on new observations—I think you would call this “low metaconfidence.” But your strategy for updating based on new evidence is already encoded in your priors for what evidence you expect to observe. I don’t think metaconfidence is useful as a new, independent concept, apart from indicating that your probabilities are susceptible to change based on future evidence.
In particular, if “metaconfidence” considerations seem to be prompting you to distrust your probability as being either systematically too high or too low, then you should just immediately update your probability in that direction, by conservation of expected evidence. So when you say
the pony is less likely, and the sunrise is more likely, than a naive probability estimate would suggest.
then if you really believe that, you ought to preemptively update your “naive” probabilities until you no longer believe they are systematically biased.
I think a particular misconception may be leading you astray:
When we talk to the crazy pony man, and to the woman with the coin, what we leave with are two identical numerical probabilities. However, those numbers do not represent the sum total of the information at our disposal. In the two cases, we have differing levels of confidence in our levels of confidence. And, furthermore, this difference has an actual ramifications on what a rational agent should expect to observe.
I think what you are referring to here is that if you repeated the 100-coin-flip experiment 2^100 times, you expect to see 100 heads on average once. But even if the pony guy tried 2^100 times, you do not expect an average of one pony to fall from the sky. You expect him to simply lack the power, and so expect him to fail every time.
The real difference here is not metaconfidence, it’s that each set of 100 coin flips is completely independent of any other set of 100 coin flips. But the pony guy’s first attempt to summon a pony is not independent of his second attempt to summon a pony. If he fails on his first attempt, you strongly expect him to fail on every future attempt.
In more detail:
If you are assigning a probability of 1/2^100 to the pony guy’s claim, you think that the claim is pretty improbable. How improbable? You are saying, “I expect that if I heard 2^100 similarly improbable, but completely independent and uncorrelated claims, on average one such claim would be true and all the rest would be false.” One such completely independent claim to which you might assign the same probability might be if I predicted that “tomorrow, physicists will discover a new sentient fundamental particle that vacations in Bermuda every August.” If your probability is 1/2^100, you really should expect that if you heard 2^100 equally astonishing claims, one of them would actually turn out to be true. But in particular you don’t expect that if the pony guy repeated his claim 2^100 times, on average he’d be telling the truth once. Repetitions of the same claim by the same person are clearly not independent claims.
Superficially this may look different from the case of the coin flips. There, you expect that if you make 2^100 tests of the claim “flipping this coin 100 times will give all heads,” it would be true on average once. To symmetrize the situation, let me make the coin claim more specific without changing it in any meaningful way: “The first 100 flips of this coin will all be heads.” This, like “I can summon ponies from the sky” is a claim that is simply either true or false. When you assign a probability of 1/2^100 to the coin claim, you are again saying, “I expect that if I heard 2^100 similarly improbable, but completely independent and uncorrelated claims, on average one of them would be true.” Because different flips of the same coin happen to have the property of statistical independence, similarly improbable but completely independent claims are easy to construct. For instance, “The second 100 flips of this coin will all be heads,” etc.
This is all to say that I disagree that
the probability value in the second case, while superficially identical to the probability value in the first case, represents a fundamentally different kind of claim about reality than the first case.
The two probabilities represent exactly analogous claims, and there is no need for a notion of metaconfidence to distinguish them.
I am skeptical about “metaconfidence.”
If you have all the relevant probabilities, you have all the information you need to calculate expected utilities for all possible choices—you don’t need to decide on how “metaconfident” you are in these probabilities. Your probabilities may not be based on very good data, and so you might anticipate that these probabilities will change drastically when you update on new observations—I think you would call this “low metaconfidence.” But your strategy for updating based on new evidence is already encoded in your priors for what evidence you expect to observe. I don’t think metaconfidence is useful as a new, independent concept, apart from indicating that your probabilities are susceptible to change based on future evidence.
In particular, if “metaconfidence” considerations seem to be prompting you to distrust your probability as being either systematically too high or too low, then you should just immediately update your probability in that direction, by conservation of expected evidence. So when you say
then if you really believe that, you ought to preemptively update your “naive” probabilities until you no longer believe they are systematically biased.
I think a particular misconception may be leading you astray:
I think what you are referring to here is that if you repeated the 100-coin-flip experiment 2^100 times, you expect to see 100 heads on average once. But even if the pony guy tried 2^100 times, you do not expect an average of one pony to fall from the sky. You expect him to simply lack the power, and so expect him to fail every time.
The real difference here is not metaconfidence, it’s that each set of 100 coin flips is completely independent of any other set of 100 coin flips. But the pony guy’s first attempt to summon a pony is not independent of his second attempt to summon a pony. If he fails on his first attempt, you strongly expect him to fail on every future attempt.
In more detail:
If you are assigning a probability of 1/2^100 to the pony guy’s claim, you think that the claim is pretty improbable. How improbable? You are saying, “I expect that if I heard 2^100 similarly improbable, but completely independent and uncorrelated claims, on average one such claim would be true and all the rest would be false.” One such completely independent claim to which you might assign the same probability might be if I predicted that “tomorrow, physicists will discover a new sentient fundamental particle that vacations in Bermuda every August.” If your probability is 1/2^100, you really should expect that if you heard 2^100 equally astonishing claims, one of them would actually turn out to be true. But in particular you don’t expect that if the pony guy repeated his claim 2^100 times, on average he’d be telling the truth once. Repetitions of the same claim by the same person are clearly not independent claims.
Superficially this may look different from the case of the coin flips. There, you expect that if you make 2^100 tests of the claim “flipping this coin 100 times will give all heads,” it would be true on average once. To symmetrize the situation, let me make the coin claim more specific without changing it in any meaningful way: “The first 100 flips of this coin will all be heads.” This, like “I can summon ponies from the sky” is a claim that is simply either true or false. When you assign a probability of 1/2^100 to the coin claim, you are again saying, “I expect that if I heard 2^100 similarly improbable, but completely independent and uncorrelated claims, on average one of them would be true.” Because different flips of the same coin happen to have the property of statistical independence, similarly improbable but completely independent claims are easy to construct. For instance, “The second 100 flips of this coin will all be heads,” etc.
This is all to say that I disagree that
The two probabilities represent exactly analogous claims, and there is no need for a notion of metaconfidence to distinguish them.