I am not sure that that means. Example: I claim that this coin is biased. I do a hundred coin flips, it comes up heads 55 times. Is this “clear evidence”?
Without crunching the numbers, my best guess is no, a fair coin is not very unlikely to come up heads 55 times out of 100. I would guess that no possible P(heads) would have a likelihood ratio much greater than 1 from that test. If one of the hypotheses is that the coin is unfair in a way that causes it to always get exactly 55 heads in 100 flips, that might be clear/strong evidence, but this would require a different mechanism than usually implied when discussing coin flips.
Does it ever get strong enough for you to dismiss all claimed evidence of paranormal powers sight unseen? I don’t know—it depends on your prior and on how did you update. I expect different results with different people.
I don’t know either. This is a rather different question from whether you’re getting evidence at all, though.
No need for best guesses—this is a standard problem in statistics. What it boils down to is that there is a specific distribution of the number of heads that 100 tosses of a fair coin would produce. You look at this distribution, note where 55 heads are on it… and then? What is clear evidence? how high a probability number makes things “likely” or “unlikely”? It’s up to you to decide what level of certainty is acceptable to you.
The Bayesian approach, of course, sidesteps all this and just updates the belief. The downside is that the output you get is not a simple “likely” or “unlikely”, it’s a full distribution and it’s still up to you what to make out of it.
...whether you’re getting evidence at all
As I said, it’s complicated and, in particular, depends on the specifics of the belief you’re interested in.
I would expect it to be hard to get to high levels of certainty in beliefs of the type “It’s impossible to do X” unless there are e.g. obvious physical constraints.
No need for best guesses—this is a standard problem in statistics. What it boils down to is that there is a specific distribution of the number of heads that 100 tosses of a fair coin would produce. You look at this distribution, note where 55 heads are on it… and then? What is clear evidence? how high a probability number makes things “likely” or “unlikely”? It’s up to you to decide what level of certainty is acceptable to you.
The Bayesian approach, of course, sidesteps all this and just updates the belief. The downside is that the output you get is not a simple “likely” or “unlikely”, it’s a full distribution and it’s still up to you what to make out of it.
Right, it’s definitely not a hard problem to calculate directly; I specifically chose not to do so, because I don’t think you need to run the numbers here to know roughly what they’ll look like. Specifically, this test shouldn’t yield even a 2:1 likelihood ratio for any specific P(heads):fair coin, and it’s only one standard deviation from the mean. Either way, it doesn’t give us much confidence that the coin isn’t fair.
Asking what is clear evidence sounds to me like asking what is hot water; it’s a quantitative thing which we describe with qualitative words. 55 heads is not very clear; 56 would be a little clearer; 100 heads is much clearer, but still not perfectly so.
Without crunching the numbers, my best guess is no, a fair coin is not very unlikely to come up heads 55 times out of 100. I would guess that no possible P(heads) would have a likelihood ratio much greater than 1 from that test.
If one of the hypotheses is that the coin is unfair in a way that causes it to always get exactly 55 heads in 100 flips, that might be clear/strong evidence, but this would require a different mechanism than usually implied when discussing coin flips.
I don’t know either. This is a rather different question from whether you’re getting evidence at all, though.
No need for best guesses—this is a standard problem in statistics. What it boils down to is that there is a specific distribution of the number of heads that 100 tosses of a fair coin would produce. You look at this distribution, note where 55 heads are on it… and then? What is clear evidence? how high a probability number makes things “likely” or “unlikely”? It’s up to you to decide what level of certainty is acceptable to you.
The Bayesian approach, of course, sidesteps all this and just updates the belief. The downside is that the output you get is not a simple “likely” or “unlikely”, it’s a full distribution and it’s still up to you what to make out of it.
As I said, it’s complicated and, in particular, depends on the specifics of the belief you’re interested in.
I would expect it to be hard to get to high levels of certainty in beliefs of the type “It’s impossible to do X” unless there are e.g. obvious physical constraints.
Right, it’s definitely not a hard problem to calculate directly; I specifically chose not to do so, because I don’t think you need to run the numbers here to know roughly what they’ll look like. Specifically, this test shouldn’t yield even a 2:1 likelihood ratio for any specific P(heads):fair coin, and it’s only one standard deviation from the mean. Either way, it doesn’t give us much confidence that the coin isn’t fair.
Asking what is clear evidence sounds to me like asking what is hot water; it’s a quantitative thing which we describe with qualitative words. 55 heads is not very clear; 56 would be a little clearer; 100 heads is much clearer, but still not perfectly so.