What does it mean for a probability not to be well defined in this context? I mean, I think I share the intuition, but I’m not really comfortable with it either. Doesn’t it seem strange that a probability could be well defined until I start learning more about it and trying to change it? How little do I have to care about the probability before it becomes well defined again?
As soon as the oracle is trying to make predictions that are affected by what the oracle says, the problem she has to solve shifts from “estimate the probabilities” to “choose what information to give, so as to produce consistent results given how that information will affect what happens”. In some cases there might not be anything she can say that yields consistent results. Exactly where (if at all) that becomes impossible depends on the details of the situation.
What does it mean for a probability not to be well defined in this context? I mean, I think I share the intuition, but I’m not really comfortable with it either. Doesn’t it seem strange that a probability could be well defined until I start learning more about it and trying to change it? How little do I have to care about the probability before it becomes well defined again?
As soon as the oracle is trying to make predictions that are affected by what the oracle says, the problem she has to solve shifts from “estimate the probabilities” to “choose what information to give, so as to produce consistent results given how that information will affect what happens”. In some cases there might not be anything she can say that yields consistent results. Exactly where (if at all) that becomes impossible depends on the details of the situation.