Yeah, I think I described P(A|B) when trying to describe the sensitivity, you are right that whether aliens actually abduct people given Bob experienced aliens abducting him is P(A|B). It’s possible I need to retract the whole section and example.
I agree. But I don’t think that you should discard the text entirely, because it seems to me that there is actually a lesson here.
I have had this experience many times: someone (sometimes on this very website) will say something like, “I know for a fact that X; my experience proves it to me beyond any doubt; I accept that my account of it won’t convince you of X, but I at least am certain of it”.
And what I often think in such cases (but perhaps too rarely say) is:
“But you shouldn’t be certain of it. It’s not just that I don’t believe X, merely based on your experience. It’s that you shouldn’t believe X, merely based on your experience. You, yourself, have not seen nearly enough evidence to convince you of X—if you were being a proper Bayesian about it. Not just my, but your conclusion, should be that, actually, X is probably false. Your experience is insufficient to convince me, but it should not have convinced you, either!”
(This is related to something that Robyn Dawes talks about in Rational Choice in an Uncertain World, when he says that people are often too eager to learn from experience.)
This is also related to what E. T. Jaynes calls “resurrection of dead hypotheses”. If you have an alien abduction experience, then this should indeed raise your probability estimate of aliens existing and abducting people. But it should also raise your probability estimate of you being crazy and having hallucinations (to take one example). And since the latter was much more probable than the former to begin with, and the evidence was compatible with both possibilities, observing the evidence cannot result in our coming to believe the former rather than the latter. As Jaynes says (in reference to his example of whether evidence of psychic powers should make one believe in psychic powers):
…Indeed, the very evidence which the ESPers throw at us to convince us, has the opposite effect on our state of belief; issuing reports of sensational data defeats its own purpose. For if the prior probability of deception is greater than that of ESP, then the more improbable the alleged data are on the null hypothesis of no deception and no ESP, the more strongly we are led to believe, not in ESP, but in deception. For this reason, the advocates of ESP (or any other marvel) will never succeed in persuading scientists that their phenomenon is real, until they learn how to eliminate the possibility of deception in the mind of the reader.
I agree. But I don’t think that you should discard the text entirely, because it seems to me that there is actually a lesson here.
I have had this experience many times: someone (sometimes on this very website) will say something like, “I know for a fact that X; my experience proves it to me beyond any doubt; I accept that my account of it won’t convince you of X, but I at least am certain of it”.
And what I often think in such cases (but perhaps too rarely say) is:
“But you shouldn’t be certain of it. It’s not just that I don’t believe X, merely based on your experience. It’s that you shouldn’t believe X, merely based on your experience. You, yourself, have not seen nearly enough evidence to convince you of X—if you were being a proper Bayesian about it. Not just my, but your conclusion, should be that, actually, X is probably false. Your experience is insufficient to convince me, but it should not have convinced you, either!”
(EDIT: For example.)
(This is related to something that Robyn Dawes talks about in Rational Choice in an Uncertain World, when he says that people are often too eager to learn from experience.)
This is also related to what E. T. Jaynes calls “resurrection of dead hypotheses”. If you have an alien abduction experience, then this should indeed raise your probability estimate of aliens existing and abducting people. But it should also raise your probability estimate of you being crazy and having hallucinations (to take one example). And since the latter was much more probable than the former to begin with, and the evidence was compatible with both possibilities, observing the evidence cannot result in our coming to believe the former rather than the latter. As Jaynes says (in reference to his example of whether evidence of psychic powers should make one believe in psychic powers):