Again, I cannot see how you can observe nothing and call it evidence. It is semantics, really, since it makes no difference for the equations, but it makes ~E a positive observation of something and E a negative observation, which is, to me, silly.
Think of ~E as meaning “We observed something, but that ‘something’ was something other than subversive activity. That is, what we observed was a member of the class of all things that aren’t subversive activity.”
Is this still “silly”?
Yes. Though, again, I’d rather R be “There is a 5th column” to keep it from being confusing.
This wouldn’t change what Warren is saying. It would only change the symbols that we use to restate what he is saying. We would now write ~R to mean “There is no fifth column”. So Warren’s claim, on my reading, would be
p(Q|E) > P(Q) and p(~R|E) > p(~R).
That is, I would just replace “R” everywhere with “~R”. Why is this less confusing? Not observing subversive activity is evidence for there being no fifth column. But it is also evidence for there being a fifth column that is marshaling its resources for a Pearl-Harbor type attack.
Maybe all of these double-negatives are confusing, but that is what the propositional calculus is for: it makes it easy to juggle the negatives just like negation signs in algebra.
My biggest problem with calling a lack of evidence evidence is that it is unnecessary in the first place, which makes it confusing when it comes to discussing it.
Also, I’m not arguing for or against the existence of the fifth column. I think I was unclear about that earlier, and I think we probably got a signal or two crossed. The fifth column was a fact, it existed in Japan, and it is the reason they were afraid of a fifth column in America.
Warren also never argued their existence, only their activity, so I don’t see why you have a Q and an R at all. Re-read the statement, he took the 5th column’s existence as a given.
What I’m arguing is the idea that a lack of evidence of subversive activity can be strong evidence that a plan similar to Pearl Harbor is being hatched.
To that end, I went ahead and made some calculations.
These are my assumptions, and I feel they are historically reasonable (I didn’t cite studies, so I can’t exactly call them accurate):
1% of all subversive plots are surprise plots (a-la Pearl Harbor). I call these p(subversion).
Evidence for such plots I call p(evidence).
90% of the time when there is such a plot, there is evidence of it before the fact. I call this p(evidence|subversion).
This is the critical part of Warren’s statement—he is essentially assuming the opposite of what I say here, and I assert this is not reasonable given what we know of such plots. There was even evidence of the Pearl Harbor plot before hand. An attack was expected and planned for; it was really only the location (and the lack of a prior declaration) and precise timing that was a surprise militarily. I’ve frankly never heard of a case of a surprise attack with absolutely no evidence that it would occur, so I believe I am being extremely generous with this number. I would not accept lowering this number much further than this.
Last, I assert that 5% of the time when evidence is found for subversion, no subversion actually occurs. Again, I think this is a reasonable number, and probably too low. I wouldn’t have a problem adjusting this number down as low as 1%.
Everything else is calculated based on these three assumptions.
So my conclusion on the question of how likely a lack of evidence implies a plot for subversion is drawn from the last two figures. Given my assumptions, which I believe are consistent with history, 99.89% of the time when there is no evidence of a plot for a surprise attack, there is no plot for a suprise attack. This means 0.11% of the time when there is no evidence of a plot, there actually is a plot.
Thus, likelihood of a Pearl Harbor style plot when there is no evidence to that fact is 0.11%.
It looks like our views have converged. What you wrote above seems to be in agreement with what I wrote here:
What Warren said is consistent with coherent Bayesian updating, even if he was updating on a bizarre prior. It might have been wrong to put a high prior probability on subversive activity, but the probability calculus doesn’t tell you how to pick your prior. All I am saying is that the Warren quote, in and of itself, does not constitute a violation of the rules of the probability calculus.
Maybe Warren committed such a violation earlier on. Maybe that’s how he arrived at such a high prior for the existence of subversive activity. But those earlier steps in his reasoning aren’t laid out before us here, so we can’t point to any specific misapplication of Bayes’s rule, as Eliezer tried to do.
The priors that you use in your calculations look approximately right to me. Warren evidently arrived at different numbers prior to the reasoning that Eliezer quoted, so I agree that he probably made some kind of Bayesian error to get to that point. But I would be hard pressed to say exactly why your numbers seem right to me, so I can’t point to exactly where Warren made his mistake. Whatever his mistake was, it was made prior to the reasoning that Eliezer quoted.
The upshot is that we do not have this nice real-life single-paragraph encapsulation of mathematically fallacious Bayesian reasoning.
Think of ~E as meaning “We observed something, but that ‘something’ was something other than subversive activity. That is, what we observed was a member of the class of all things that aren’t subversive activity.”
Is this still “silly”?
This wouldn’t change what Warren is saying. It would only change the symbols that we use to restate what he is saying. We would now write ~R to mean “There is no fifth column”. So Warren’s claim, on my reading, would be
p(Q|E) > P(Q) and p(~R|E) > p(~R).
That is, I would just replace “R” everywhere with “~R”. Why is this less confusing? Not observing subversive activity is evidence for there being no fifth column. But it is also evidence for there being a fifth column that is marshaling its resources for a Pearl-Harbor type attack.
Maybe all of these double-negatives are confusing, but that is what the propositional calculus is for: it makes it easy to juggle the negatives just like negation signs in algebra.
My biggest problem with calling a lack of evidence evidence is that it is unnecessary in the first place, which makes it confusing when it comes to discussing it.
Also, I’m not arguing for or against the existence of the fifth column. I think I was unclear about that earlier, and I think we probably got a signal or two crossed. The fifth column was a fact, it existed in Japan, and it is the reason they were afraid of a fifth column in America.
Warren also never argued their existence, only their activity, so I don’t see why you have a Q and an R at all. Re-read the statement, he took the 5th column’s existence as a given.
What I’m arguing is the idea that a lack of evidence of subversive activity can be strong evidence that a plan similar to Pearl Harbor is being hatched.
To that end, I went ahead and made some calculations.
These are my assumptions, and I feel they are historically reasonable (I didn’t cite studies, so I can’t exactly call them accurate):
1% of all subversive plots are surprise plots (a-la Pearl Harbor). I call these p(subversion).
Evidence for such plots I call p(evidence).
90% of the time when there is such a plot, there is evidence of it before the fact. I call this p(evidence|subversion).
This is the critical part of Warren’s statement—he is essentially assuming the opposite of what I say here, and I assert this is not reasonable given what we know of such plots. There was even evidence of the Pearl Harbor plot before hand. An attack was expected and planned for; it was really only the location (and the lack of a prior declaration) and precise timing that was a surprise militarily. I’ve frankly never heard of a case of a surprise attack with absolutely no evidence that it would occur, so I believe I am being extremely generous with this number. I would not accept lowering this number much further than this.
Last, I assert that 5% of the time when evidence is found for subversion, no subversion actually occurs. Again, I think this is a reasonable number, and probably too low. I wouldn’t have a problem adjusting this number down as low as 1%.
Everything else is calculated based on these three assumptions.
p(subversion) = 1% p(~subversion) = 99%
p(evidence|subversion) = 90% p(~evidence|subversion) = 10% p(evidence|~subversion) = 4.95% p(~evidence|~subversion) = 94.05%
p(evidence) = 5.85% p(~evidence) = 94.15%
p(subversion|evidence) = 15.52% p(~subversion|evidence) = 35.35% p(subversion|~evidence) = 0.11% p(~subversion|~evidence) = 99.89%
So my conclusion on the question of how likely a lack of evidence implies a plot for subversion is drawn from the last two figures. Given my assumptions, which I believe are consistent with history, 99.89% of the time when there is no evidence of a plot for a surprise attack, there is no plot for a suprise attack. This means 0.11% of the time when there is no evidence of a plot, there actually is a plot.
Thus, likelihood of a Pearl Harbor style plot when there is no evidence to that fact is 0.11%.
It looks like our views have converged. What you wrote above seems to be in agreement with what I wrote here:
The priors that you use in your calculations look approximately right to me. Warren evidently arrived at different numbers prior to the reasoning that Eliezer quoted, so I agree that he probably made some kind of Bayesian error to get to that point. But I would be hard pressed to say exactly why your numbers seem right to me, so I can’t point to exactly where Warren made his mistake. Whatever his mistake was, it was made prior to the reasoning that Eliezer quoted.
The upshot is that we do not have this nice real-life single-paragraph encapsulation of mathematically fallacious Bayesian reasoning.