We could also try to summarize some features of such epistemic states by talking about the instability of estimates—the degree to which they are easily updated by knowledge of other events
Yes, this is Jaynes’ A_p approach.
this will be a derived feature of the probability distribution, rather than an ontologically extra feature of probability.
I’m not sure I follow this. There is no prior distribution for the per-coin payout probabilities that can accurately reflect all our knowledge.
I reject that this is a good reason for probability theorists to panic.
Yes, it’s clear from comments that my OP was somewhat misleading as to its purpose. Overall, the sequence intends to discuss cases of uncertainty in which probability theory is the wrong tool for the job, and what to do instead.
However, this particular article intended only to introduce the idea that one’s confidence in a probability estimate is independent from that estimate, and to develop the A_p (meta-probability) approach to expressing that confidence.
I’m not sure I follow this. There is no prior distribution for the per-coin payout probabilities that can accurately reflect all our knowledge.
Are we talking about the Laplace vs. fair coins? Are you claiming there’s no prior distribution over sequences which reflects our knowledge? If so I think you are wrong as a matter of math.
Are you claiming there’s no prior distribution over sequences which reflects our knowledge?
No. Well, not so long as we’re allowed to take our own actions into account!
I want to emphasize—since many commenters seem to have mistaken me on this—that there’s an obvious, correct solution to this problem (which I made explicit in the OP). I deliberately made the problem as simple as possible in order to present the A_p framework clearly.
Are you claiming there’s no prior distribution over sequences which reflects our knowledge?
No. Well, not so long as we’re allowed to take our own actions into account!
Heh! Yes, traditional causal models have structure beyond what is present in the corresponding probability distribution over those models, though this has to do with computing counterfactuals rather than meta-probability or estimate instability. Work continues at MIRI decision theory workshops on the search for ways to turn some of this back into probability, but yes, in my world causal models are things we assign probabilities to, over and beyond probabilities we assign to joint collections of events. They are still models of reality to which a probability is assigned, though. (See Judea Pearl’s “Why I Am Only A Half-Bayesian”.)
I don’t really understand what “being Bayesian about causal models” means. What makes the most sense (e.g. what people typicalliy do) is:
(a) “be Bayesian about statistical models”, and
(b) Use additional assumptions to interpret the output of (a) causally.
(a) makes sense because I understand how evidence help me select among sets of statistical alternatives.
(b) also makes sense, but then no one will accept your answer without actually verifying the causal model by experiment—because your assumptions linking the statistical model to a causal one may not be true. And this game of verifying these assumptions doesn’t seem like a Bayesian kind of game at all.
I don’t know what it means to use Bayes theorem to select among causal models directly.
It means that you figure out which causal models look more or less like what you observed.
More generally: There’s a language of causal models which, we think, allows us to describe the actual universe, and many other universes besides. Some of these models are simpler than others. Any given sequence of experiences has some probability of being encountered in a given causal universe.
Yes, this is Jaynes’ A_p approach.
I’m not sure I follow this. There is no prior distribution for the per-coin payout probabilities that can accurately reflect all our knowledge.
Yes, it’s clear from comments that my OP was somewhat misleading as to its purpose. Overall, the sequence intends to discuss cases of uncertainty in which probability theory is the wrong tool for the job, and what to do instead.
However, this particular article intended only to introduce the idea that one’s confidence in a probability estimate is independent from that estimate, and to develop the A_p (meta-probability) approach to expressing that confidence.
Are we talking about the Laplace vs. fair coins? Are you claiming there’s no prior distribution over sequences which reflects our knowledge? If so I think you are wrong as a matter of math.
No. Well, not so long as we’re allowed to take our own actions into account!
I want to emphasize—since many commenters seem to have mistaken me on this—that there’s an obvious, correct solution to this problem (which I made explicit in the OP). I deliberately made the problem as simple as possible in order to present the A_p framework clearly.
Not sure what you are asking here, sorry...
Heh! Yes, traditional causal models have structure beyond what is present in the corresponding probability distribution over those models, though this has to do with computing counterfactuals rather than meta-probability or estimate instability. Work continues at MIRI decision theory workshops on the search for ways to turn some of this back into probability, but yes, in my world causal models are things we assign probabilities to, over and beyond probabilities we assign to joint collections of events. They are still models of reality to which a probability is assigned, though. (See Judea Pearl’s “Why I Am Only A Half-Bayesian”.)
I don’t really understand what “being Bayesian about causal models” means. What makes the most sense (e.g. what people typicalliy do) is:
(a) “be Bayesian about statistical models”, and
(b) Use additional assumptions to interpret the output of (a) causally.
(a) makes sense because I understand how evidence help me select among sets of statistical alternatives.
(b) also makes sense, but then no one will accept your answer without actually verifying the causal model by experiment—because your assumptions linking the statistical model to a causal one may not be true. And this game of verifying these assumptions doesn’t seem like a Bayesian kind of game at all.
I don’t know what it means to use Bayes theorem to select among causal models directly.
It means that you figure out which causal models look more or less like what you observed.
More generally: There’s a language of causal models which, we think, allows us to describe the actual universe, and many other universes besides. Some of these models are simpler than others. Any given sequence of experiences has some probability of being encountered in a given causal universe.