I don’t like the way you describe that. It is confusing. The evidence is subversive activity. You cannot go out and look for no subversive activity, that makes no sense.
Our evidence is always only what we have observed. Maybe it is strange to say that you “looked for no subversive activity”. But you certainly can look for subversive activity and fail to find it. Not seeing subversive activity when you looked for it is Bayesian evidence. But it would be an error to condition on there being no subversive activity at all, even hidden activity. That would be going beyond your observations. You can only condition on what you saw or didn’t see when you looked.
Do you agree that, under some priors, you could have p(Q|E) > p(Q) and p(R|E) > p(R), even though Q implies not-R?
Absolutely. I said so two posts up. The question is not about Q and R, though, it’s about Q and ~Q.
Okay, I think that we’re homing in on the nub of the disagreement.
The propositions in question are
“There is a fifth column that will coordinate a Pearl-Harbor type attack”, and
“There is no fifth column”.
These are Q and R, respectively. They are not negations of each other. Do you agree?
Again, I cannot see how you can observe nothing and call it evidence. It is semantics, really, since it makes no difference for the equations, but it makes ~E a positive observation of something and E a negative observation, which is, to me, silly.
These are Q and R, respectively. They are not negations of each other. Do you agree?
Yes. Though, again, I’d rather R be “There is a 5th column” to keep it from being confusing.
With Q there was no evidence that the fifth column was coordinating a timed attack, yet Warren’s strongest evidence for it was that there was no evidence for it.
Pearl harbor types of evidence are black swans. You can’t just pull them out of the air and add them to your reasoning when you have no solid justification for it. There are a billion other black swans he could have used—what if the Japanese are actually all vampires and had designs on draining the Americans dry? You’ve got no evidence they aren’t, so clearly they are just biding their time!
The former is slightly more reasonable, since something similar had happened recently (though in an entirely different context), but it is no more justified as evidence than the evidence in the vampire scenario.
You must look for other evidence that suggests the 5th column was planning an attack, the fact that you have not been attacked yet is not in any way evidence that they are planning an attack. It is only really evidence that, if they were planning something, they hadn’t done it yet. That’s all you can get from that—just a guess.
To that end, Warren had no evidence that an American chapter of the 5th Column even existed. There was secret evidence to that effect, but he was not privy to it. He was making the whole thing up because he was afraid.
It was completely unjustified.
Besides, it doesn’t make sense. Timed attacks are designed to catch you off guard. After Pearl Harbor, people were always on guard. It wouldn’t have had the same effect; a much more effective strategy would have been smaller, guerrilla-type sabotages from within, which they also had zero evidence of.
Again, I cannot see how you can observe nothing and call it evidence. It is semantics, really, since it makes no difference for the equations, but it makes ~E a positive observation of something and E a negative observation, which is, to me, silly.
Think of ~E as meaning “We observed something, but that ‘something’ was something other than subversive activity. That is, what we observed was a member of the class of all things that aren’t subversive activity.”
Is this still “silly”?
Yes. Though, again, I’d rather R be “There is a 5th column” to keep it from being confusing.
This wouldn’t change what Warren is saying. It would only change the symbols that we use to restate what he is saying. We would now write ~R to mean “There is no fifth column”. So Warren’s claim, on my reading, would be
p(Q|E) > P(Q) and p(~R|E) > p(~R).
That is, I would just replace “R” everywhere with “~R”. Why is this less confusing? Not observing subversive activity is evidence for there being no fifth column. But it is also evidence for there being a fifth column that is marshaling its resources for a Pearl-Harbor type attack.
Maybe all of these double-negatives are confusing, but that is what the propositional calculus is for: it makes it easy to juggle the negatives just like negation signs in algebra.
My biggest problem with calling a lack of evidence evidence is that it is unnecessary in the first place, which makes it confusing when it comes to discussing it.
Also, I’m not arguing for or against the existence of the fifth column. I think I was unclear about that earlier, and I think we probably got a signal or two crossed. The fifth column was a fact, it existed in Japan, and it is the reason they were afraid of a fifth column in America.
Warren also never argued their existence, only their activity, so I don’t see why you have a Q and an R at all. Re-read the statement, he took the 5th column’s existence as a given.
What I’m arguing is the idea that a lack of evidence of subversive activity can be strong evidence that a plan similar to Pearl Harbor is being hatched.
To that end, I went ahead and made some calculations.
These are my assumptions, and I feel they are historically reasonable (I didn’t cite studies, so I can’t exactly call them accurate):
1% of all subversive plots are surprise plots (a-la Pearl Harbor). I call these p(subversion).
Evidence for such plots I call p(evidence).
90% of the time when there is such a plot, there is evidence of it before the fact. I call this p(evidence|subversion).
This is the critical part of Warren’s statement—he is essentially assuming the opposite of what I say here, and I assert this is not reasonable given what we know of such plots. There was even evidence of the Pearl Harbor plot before hand. An attack was expected and planned for; it was really only the location (and the lack of a prior declaration) and precise timing that was a surprise militarily. I’ve frankly never heard of a case of a surprise attack with absolutely no evidence that it would occur, so I believe I am being extremely generous with this number. I would not accept lowering this number much further than this.
Last, I assert that 5% of the time when evidence is found for subversion, no subversion actually occurs. Again, I think this is a reasonable number, and probably too low. I wouldn’t have a problem adjusting this number down as low as 1%.
Everything else is calculated based on these three assumptions.
So my conclusion on the question of how likely a lack of evidence implies a plot for subversion is drawn from the last two figures. Given my assumptions, which I believe are consistent with history, 99.89% of the time when there is no evidence of a plot for a surprise attack, there is no plot for a suprise attack. This means 0.11% of the time when there is no evidence of a plot, there actually is a plot.
Thus, likelihood of a Pearl Harbor style plot when there is no evidence to that fact is 0.11%.
It looks like our views have converged. What you wrote above seems to be in agreement with what I wrote here:
What Warren said is consistent with coherent Bayesian updating, even if he was updating on a bizarre prior. It might have been wrong to put a high prior probability on subversive activity, but the probability calculus doesn’t tell you how to pick your prior. All I am saying is that the Warren quote, in and of itself, does not constitute a violation of the rules of the probability calculus.
Maybe Warren committed such a violation earlier on. Maybe that’s how he arrived at such a high prior for the existence of subversive activity. But those earlier steps in his reasoning aren’t laid out before us here, so we can’t point to any specific misapplication of Bayes’s rule, as Eliezer tried to do.
The priors that you use in your calculations look approximately right to me. Warren evidently arrived at different numbers prior to the reasoning that Eliezer quoted, so I agree that he probably made some kind of Bayesian error to get to that point. But I would be hard pressed to say exactly why your numbers seem right to me, so I can’t point to exactly where Warren made his mistake. Whatever his mistake was, it was made prior to the reasoning that Eliezer quoted.
The upshot is that we do not have this nice real-life single-paragraph encapsulation of mathematically fallacious Bayesian reasoning.
Our evidence is always only what we have observed. Maybe it is strange to say that you “looked for no subversive activity”. But you certainly can look for subversive activity and fail to find it. Not seeing subversive activity when you looked for it is Bayesian evidence. But it would be an error to condition on there being no subversive activity at all, even hidden activity. That would be going beyond your observations. You can only condition on what you saw or didn’t see when you looked.
Okay, I think that we’re homing in on the nub of the disagreement.
The propositions in question are
“There is a fifth column that will coordinate a Pearl-Harbor type attack”, and
“There is no fifth column”.
These are Q and R, respectively. They are not negations of each other. Do you agree?
Again, I cannot see how you can observe nothing and call it evidence. It is semantics, really, since it makes no difference for the equations, but it makes ~E a positive observation of something and E a negative observation, which is, to me, silly.
Yes. Though, again, I’d rather R be “There is a 5th column” to keep it from being confusing.
With Q there was no evidence that the fifth column was coordinating a timed attack, yet Warren’s strongest evidence for it was that there was no evidence for it.
Pearl harbor types of evidence are black swans. You can’t just pull them out of the air and add them to your reasoning when you have no solid justification for it. There are a billion other black swans he could have used—what if the Japanese are actually all vampires and had designs on draining the Americans dry? You’ve got no evidence they aren’t, so clearly they are just biding their time!
The former is slightly more reasonable, since something similar had happened recently (though in an entirely different context), but it is no more justified as evidence than the evidence in the vampire scenario.
You must look for other evidence that suggests the 5th column was planning an attack, the fact that you have not been attacked yet is not in any way evidence that they are planning an attack. It is only really evidence that, if they were planning something, they hadn’t done it yet. That’s all you can get from that—just a guess.
To that end, Warren had no evidence that an American chapter of the 5th Column even existed. There was secret evidence to that effect, but he was not privy to it. He was making the whole thing up because he was afraid.
It was completely unjustified.
Besides, it doesn’t make sense. Timed attacks are designed to catch you off guard. After Pearl Harbor, people were always on guard. It wouldn’t have had the same effect; a much more effective strategy would have been smaller, guerrilla-type sabotages from within, which they also had zero evidence of.
He didn’t observe “nothing.” He observed factories and shipyards and so forth, continuing to operate without apparent sabotage.
Think of ~E as meaning “We observed something, but that ‘something’ was something other than subversive activity. That is, what we observed was a member of the class of all things that aren’t subversive activity.”
Is this still “silly”?
This wouldn’t change what Warren is saying. It would only change the symbols that we use to restate what he is saying. We would now write ~R to mean “There is no fifth column”. So Warren’s claim, on my reading, would be
p(Q|E) > P(Q) and p(~R|E) > p(~R).
That is, I would just replace “R” everywhere with “~R”. Why is this less confusing? Not observing subversive activity is evidence for there being no fifth column. But it is also evidence for there being a fifth column that is marshaling its resources for a Pearl-Harbor type attack.
Maybe all of these double-negatives are confusing, but that is what the propositional calculus is for: it makes it easy to juggle the negatives just like negation signs in algebra.
My biggest problem with calling a lack of evidence evidence is that it is unnecessary in the first place, which makes it confusing when it comes to discussing it.
Also, I’m not arguing for or against the existence of the fifth column. I think I was unclear about that earlier, and I think we probably got a signal or two crossed. The fifth column was a fact, it existed in Japan, and it is the reason they were afraid of a fifth column in America.
Warren also never argued their existence, only their activity, so I don’t see why you have a Q and an R at all. Re-read the statement, he took the 5th column’s existence as a given.
What I’m arguing is the idea that a lack of evidence of subversive activity can be strong evidence that a plan similar to Pearl Harbor is being hatched.
To that end, I went ahead and made some calculations.
These are my assumptions, and I feel they are historically reasonable (I didn’t cite studies, so I can’t exactly call them accurate):
1% of all subversive plots are surprise plots (a-la Pearl Harbor). I call these p(subversion).
Evidence for such plots I call p(evidence).
90% of the time when there is such a plot, there is evidence of it before the fact. I call this p(evidence|subversion).
This is the critical part of Warren’s statement—he is essentially assuming the opposite of what I say here, and I assert this is not reasonable given what we know of such plots. There was even evidence of the Pearl Harbor plot before hand. An attack was expected and planned for; it was really only the location (and the lack of a prior declaration) and precise timing that was a surprise militarily. I’ve frankly never heard of a case of a surprise attack with absolutely no evidence that it would occur, so I believe I am being extremely generous with this number. I would not accept lowering this number much further than this.
Last, I assert that 5% of the time when evidence is found for subversion, no subversion actually occurs. Again, I think this is a reasonable number, and probably too low. I wouldn’t have a problem adjusting this number down as low as 1%.
Everything else is calculated based on these three assumptions.
p(subversion) = 1% p(~subversion) = 99%
p(evidence|subversion) = 90% p(~evidence|subversion) = 10% p(evidence|~subversion) = 4.95% p(~evidence|~subversion) = 94.05%
p(evidence) = 5.85% p(~evidence) = 94.15%
p(subversion|evidence) = 15.52% p(~subversion|evidence) = 35.35% p(subversion|~evidence) = 0.11% p(~subversion|~evidence) = 99.89%
So my conclusion on the question of how likely a lack of evidence implies a plot for subversion is drawn from the last two figures. Given my assumptions, which I believe are consistent with history, 99.89% of the time when there is no evidence of a plot for a surprise attack, there is no plot for a suprise attack. This means 0.11% of the time when there is no evidence of a plot, there actually is a plot.
Thus, likelihood of a Pearl Harbor style plot when there is no evidence to that fact is 0.11%.
It looks like our views have converged. What you wrote above seems to be in agreement with what I wrote here:
The priors that you use in your calculations look approximately right to me. Warren evidently arrived at different numbers prior to the reasoning that Eliezer quoted, so I agree that he probably made some kind of Bayesian error to get to that point. But I would be hard pressed to say exactly why your numbers seem right to me, so I can’t point to exactly where Warren made his mistake. Whatever his mistake was, it was made prior to the reasoning that Eliezer quoted.
The upshot is that we do not have this nice real-life single-paragraph encapsulation of mathematically fallacious Bayesian reasoning.