‘strong evidence is the sort of evidence we couldn’t possibly find if the hypothesis were false’.
-blink-
If you mean this, please elaborate. If not, please change the wording before you confuse the living daylights out of some poor newcomer.
Edit: I’m not nitpicking him for infinite certainty. I acknowledge it’s reasonable informally to tell me a ticket I’m thinking of buying couldn’t possibly win the lottery. That’s not what I mean. I mean even finding some overwhelmingly strong evidence doesn’t necessarily mean the hypothesis is overwhelmingly likely to be true. If the comment’s misleading then given it’s subject it seems worth pointing out!
Example: Say you’re randomly chosen to take a test with a false positive rate of 1% for a cancer that occurs in 0.1% of the population, and it returns positive. That’s strong evidence for the hypothesis that you have that cancer, but the hypothesis is probably false.
Strongly seconded. Generally, it seems to me that Eliezer frequently seriously confuses people by mixing literal statements with hyperbole like this or “shut up and do the impossible”. I definitely see the merit of the greater emotional impact, but I hope there’s some way to get it without putting off the unusually literal-minded (which I expect most people who will get anything out of OB or The Book are).
Yeah, that is kind of tricky. Let me try to explain what Eliezer_Yudkowsky meant in terms of my preferred form of the Bayes Theorem:
O(H|E) = O(H) * P(E|H) / P(E|~H)
where O indicates odds instead of probability and | indicates “given”.
In other words, “any time you observe evidence, amplify the odds you assign to your beliefs by the probability of observing the evidence if the belief were true, divided by the probabily of observing it if the belief were false.”
Also, keep in mind that Eliezer_Yudkowsky has written about how you should treat very low probability events as being “impossible”, even though you have to assign a non-zero probability to everything.
Nevertheless, his statement still isn’t literally true. The strength of the evidence depends on the ratio P(E|H)/P(E|~H), while the quoted statement only refers to the denominator. So there can be situations where you have 100:1 odds of seeing E if the hypothesis were true, but 1:1000 odds (about a 0.1% chance) of seeing E if it were false.
Such evidence is very strong—it forces you to amplify the odds you assign to H by a factor of 100,000 -- but it’s far from evidence you “couldn’t possibly find”, which to me means something like 1:10^-10 odds.
Still, Eliezer_Yudkowsky is right that, generally, strong evidence will have a very small denominator.
Strong evidence is evidence that, given certain premises, has no chance of arising.
Of course, Eliezer has also claimed that nothing can have no chance of arising (probability zero), so it’s easy to see how one might be confused about his position.
Traditionally, evidence that has less than a particular value of arising given the truth of a hypothesis (usually 5%) is considered to be strong, but that’s really an arbitrary decision.
Traditionally, evidence that has less than a particular value of arising given the truth of a hypothesis (usually 5%) is considered to be strong, but that’s really an arbitrary decision.
Correction: traditionally evidence against an hypothesis is considered strong if the chance of that evidence or any more extreme evidence arising given the truth of the hypothesis is less than an arbitrary value. (If this tradition doesn’t make sense to you, you are not alone.)
-blink-
If you mean this, please elaborate. If not, please change the wording before you confuse the living daylights out of some poor newcomer.
Edit: I’m not nitpicking him for infinite certainty. I acknowledge it’s reasonable informally to tell me a ticket I’m thinking of buying couldn’t possibly win the lottery. That’s not what I mean. I mean even finding some overwhelmingly strong evidence doesn’t necessarily mean the hypothesis is overwhelmingly likely to be true. If the comment’s misleading then given it’s subject it seems worth pointing out!
Example: Say you’re randomly chosen to take a test with a false positive rate of 1% for a cancer that occurs in 0.1% of the population, and it returns positive. That’s strong evidence for the hypothesis that you have that cancer, but the hypothesis is probably false.
Strongly seconded. Generally, it seems to me that Eliezer frequently seriously confuses people by mixing literal statements with hyperbole like this or “shut up and do the impossible”. I definitely see the merit of the greater emotional impact, but I hope there’s some way to get it without putting off the unusually literal-minded (which I expect most people who will get anything out of OB or The Book are).
Yeah, that is kind of tricky. Let me try to explain what Eliezer_Yudkowsky meant in terms of my preferred form of the Bayes Theorem:
O(H|E) = O(H) * P(E|H) / P(E|~H)
where O indicates odds instead of probability and | indicates “given”.
In other words, “any time you observe evidence, amplify the odds you assign to your beliefs by the probability of observing the evidence if the belief were true, divided by the probabily of observing it if the belief were false.”
Also, keep in mind that Eliezer_Yudkowsky has written about how you should treat very low probability events as being “impossible”, even though you have to assign a non-zero probability to everything.
Nevertheless, his statement still isn’t literally true. The strength of the evidence depends on the ratio P(E|H)/P(E|~H), while the quoted statement only refers to the denominator. So there can be situations where you have 100:1 odds of seeing E if the hypothesis were true, but 1:1000 odds (about a 0.1% chance) of seeing E if it were false.
Such evidence is very strong—it forces you to amplify the odds you assign to H by a factor of 100,000 -- but it’s far from evidence you “couldn’t possibly find”, which to me means something like 1:10^-10 odds.
Still, Eliezer_Yudkowsky is right that, generally, strong evidence will have a very small denominator.
EDIT: added link
In comments like this, we should link to the existing pages of the wiki, or create stubs of the new ones.
Bayes’ theorem on LessWrong wiki.
Strong evidence is evidence that, given certain premises, has no chance of arising.
Of course, Eliezer has also claimed that nothing can have no chance of arising (probability zero), so it’s easy to see how one might be confused about his position.
Traditionally, evidence that has less than a particular value of arising given the truth of a hypothesis (usually 5%) is considered to be strong, but that’s really an arbitrary decision.
Correction: traditionally evidence against an hypothesis is considered strong if the chance of that evidence or any more extreme evidence arising given the truth of the hypothesis is less than an arbitrary value. (If this tradition doesn’t make sense to you, you are not alone.)