The second half of the sentence was the reason I was bringing it up in this context. We’ve looked, kinda, and not very systematically, and maybe not in the right places, but haven’t found any evidence. Is it fair to call this evidence against paranormal claims?
It’s complicated, I don’t think this problem can be resolved in one or two sentences.
For example, there is clear relationship to how specific the claim/belief is. Lack of evidence is more important for very specific and easily testable claims (“I can bend this very spoon in front of your eyes”) and less important for general claims (“some people can occasionally perform telekinesis”).
Oh, and there’s lot of evidence for paranormal claims. It’s just that this evidence is contested. Some of it has been conclusively debunked, but not all.
Trying to not get sidetracked into that specific sub-discussion: should you be skeptical of any given paranormal claim (specific or general), if some people have tried but nobody has been able to produce clear evidence for it? “Clear evidence” here meaning “better evidence than we would expect if the claim is false”, per the Bayesian definition of evidence.
Should you be more or less skeptical than upon first hearing the claim, but before examining the evidence about it?
I think I’m not getting why you object to “AoE is EoA”, if appending ”...but sometimes it’s so weak that we humans can’t actually make use of it” doesn’t resolve the disagreement in much the same way that ”...but only provided you have looked, and looked in the right place” does.
“Clear evidence” here meaning “better evidence than we would expect if the claim is false”
I am not sure that that means. Example: I claim that this coin is biased. I do a hundred coin flips, it comes up heads 55 times. Is this “clear evidence”?
Should you be more or less skeptical than upon first hearing the claim, but before examining the evidence about it?
Oh-oh, that’s a question about how you should form your prior. The Bayesian approach is notoriously reticent about discussing this.
But you can think about it this way: invert the belief and make it “Everyone who claims to have paranormal powers is a fraud”. When another one is debunked, it’s positive evidence for your belief and you should update it. The more people get debunked, the stronger your belief gets.
Does it ever get strong enough for you to dismiss all claimed evidence of paranormal powers sight unseen? I don’t know—it depends on your prior and on how did you update. I expect different results with different people.
I am not sure that that means. Example: I claim that this coin is biased. I do a hundred coin flips, it comes up heads 55 times. Is this “clear evidence”?
Without crunching the numbers, my best guess is no, a fair coin is not very unlikely to come up heads 55 times out of 100. I would guess that no possible P(heads) would have a likelihood ratio much greater than 1 from that test. If one of the hypotheses is that the coin is unfair in a way that causes it to always get exactly 55 heads in 100 flips, that might be clear/strong evidence, but this would require a different mechanism than usually implied when discussing coin flips.
Does it ever get strong enough for you to dismiss all claimed evidence of paranormal powers sight unseen? I don’t know—it depends on your prior and on how did you update. I expect different results with different people.
I don’t know either. This is a rather different question from whether you’re getting evidence at all, though.
No need for best guesses—this is a standard problem in statistics. What it boils down to is that there is a specific distribution of the number of heads that 100 tosses of a fair coin would produce. You look at this distribution, note where 55 heads are on it… and then? What is clear evidence? how high a probability number makes things “likely” or “unlikely”? It’s up to you to decide what level of certainty is acceptable to you.
The Bayesian approach, of course, sidesteps all this and just updates the belief. The downside is that the output you get is not a simple “likely” or “unlikely”, it’s a full distribution and it’s still up to you what to make out of it.
...whether you’re getting evidence at all
As I said, it’s complicated and, in particular, depends on the specifics of the belief you’re interested in.
I would expect it to be hard to get to high levels of certainty in beliefs of the type “It’s impossible to do X” unless there are e.g. obvious physical constraints.
No need for best guesses—this is a standard problem in statistics. What it boils down to is that there is a specific distribution of the number of heads that 100 tosses of a fair coin would produce. You look at this distribution, note where 55 heads are on it… and then? What is clear evidence? how high a probability number makes things “likely” or “unlikely”? It’s up to you to decide what level of certainty is acceptable to you.
The Bayesian approach, of course, sidesteps all this and just updates the belief. The downside is that the output you get is not a simple “likely” or “unlikely”, it’s a full distribution and it’s still up to you what to make out of it.
Right, it’s definitely not a hard problem to calculate directly; I specifically chose not to do so, because I don’t think you need to run the numbers here to know roughly what they’ll look like. Specifically, this test shouldn’t yield even a 2:1 likelihood ratio for any specific P(heads):fair coin, and it’s only one standard deviation from the mean. Either way, it doesn’t give us much confidence that the coin isn’t fair.
Asking what is clear evidence sounds to me like asking what is hot water; it’s a quantitative thing which we describe with qualitative words. 55 heads is not very clear; 56 would be a little clearer; 100 heads is much clearer, but still not perfectly so.
The second half of the sentence was the reason I was bringing it up in this context. We’ve looked, kinda, and not very systematically, and maybe not in the right places, but haven’t found any evidence. Is it fair to call this evidence against paranormal claims?
It’s complicated, I don’t think this problem can be resolved in one or two sentences.
For example, there is clear relationship to how specific the claim/belief is. Lack of evidence is more important for very specific and easily testable claims (“I can bend this very spoon in front of your eyes”) and less important for general claims (“some people can occasionally perform telekinesis”).
Oh, and there’s lot of evidence for paranormal claims. It’s just that this evidence is contested. Some of it has been conclusively debunked, but not all.
Trying to not get sidetracked into that specific sub-discussion: should you be skeptical of any given paranormal claim (specific or general), if some people have tried but nobody has been able to produce clear evidence for it? “Clear evidence” here meaning “better evidence than we would expect if the claim is false”, per the Bayesian definition of evidence.
Should you be more or less skeptical than upon first hearing the claim, but before examining the evidence about it?
I think I’m not getting why you object to “AoE is EoA”, if appending ”...but sometimes it’s so weak that we humans can’t actually make use of it” doesn’t resolve the disagreement in much the same way that ”...but only provided you have looked, and looked in the right place” does.
I am not sure that that means. Example: I claim that this coin is biased. I do a hundred coin flips, it comes up heads 55 times. Is this “clear evidence”?
Oh-oh, that’s a question about how you should form your prior. The Bayesian approach is notoriously reticent about discussing this.
But you can think about it this way: invert the belief and make it “Everyone who claims to have paranormal powers is a fraud”. When another one is debunked, it’s positive evidence for your belief and you should update it. The more people get debunked, the stronger your belief gets.
Does it ever get strong enough for you to dismiss all claimed evidence of paranormal powers sight unseen? I don’t know—it depends on your prior and on how did you update. I expect different results with different people.
Without crunching the numbers, my best guess is no, a fair coin is not very unlikely to come up heads 55 times out of 100. I would guess that no possible P(heads) would have a likelihood ratio much greater than 1 from that test.
If one of the hypotheses is that the coin is unfair in a way that causes it to always get exactly 55 heads in 100 flips, that might be clear/strong evidence, but this would require a different mechanism than usually implied when discussing coin flips.
I don’t know either. This is a rather different question from whether you’re getting evidence at all, though.
No need for best guesses—this is a standard problem in statistics. What it boils down to is that there is a specific distribution of the number of heads that 100 tosses of a fair coin would produce. You look at this distribution, note where 55 heads are on it… and then? What is clear evidence? how high a probability number makes things “likely” or “unlikely”? It’s up to you to decide what level of certainty is acceptable to you.
The Bayesian approach, of course, sidesteps all this and just updates the belief. The downside is that the output you get is not a simple “likely” or “unlikely”, it’s a full distribution and it’s still up to you what to make out of it.
As I said, it’s complicated and, in particular, depends on the specifics of the belief you’re interested in.
I would expect it to be hard to get to high levels of certainty in beliefs of the type “It’s impossible to do X” unless there are e.g. obvious physical constraints.
Right, it’s definitely not a hard problem to calculate directly; I specifically chose not to do so, because I don’t think you need to run the numbers here to know roughly what they’ll look like. Specifically, this test shouldn’t yield even a 2:1 likelihood ratio for any specific P(heads):fair coin, and it’s only one standard deviation from the mean. Either way, it doesn’t give us much confidence that the coin isn’t fair.
Asking what is clear evidence sounds to me like asking what is hot water; it’s a quantitative thing which we describe with qualitative words. 55 heads is not very clear; 56 would be a little clearer; 100 heads is much clearer, but still not perfectly so.