I’ve never been completely happy with the “I could make 1M similar statements and be wrong once” test. It seems, I dunno, kind of a frequentist way of thinking about the probability that I’m wrong. I can’t imagine making a million statements and have no way of knowing what it’s like to feel confidence about a statement to an accuracy of one part per million.
Other ways to think of tiny probabilities:
(1) If probability theory tells me there’s a 1 in a billion chance of X happening, then P(X) is somewhere between 1 in a billion and P(I calculated wrong), the latter being much higher.
If I were running on hardware that was better at arithmetic, P(I calculated wrong) could be got down way below 1 in a billion. After all, even today’s computers do billions of arithmetic operations per second. If they had anything like a one-in-a-billion failure rate per operation, we’d find them much less useful.
(2) Think of statements like P(7 is prime) = 1 as useful simplifications. If I am examining whether 7 is prime, I wouldn’t start with a prior of 1. But if I’m testing a hypothesis about something else and it depends on (among other things) whether 7 is prime, I wouldn’t assign P(7 is prime) some ridiculously specific just-under-1 probability; I’d call it 1 and simplify the causal network accordingly.
It seems, I dunno, kind of a frequentist way of thinking about the probability that I’m wrong.
There are numerous studies that show that our brain’s natural way of thinking out probabilities is in terms of frequencies, and that people show less bias when presented with frequencies than when they are presented with percentages.
There are numerous studies that show that our brain’s natural way of thinking out probabilities is in terms of frequencies
Thinking about which probabilities?
Probability is a complex concept. The probability in the sentence “the probability of getting more than 60 heads in 100 fair coin tosses” is a very different beast from the probability in the sentence “the probability of rain tomorrow”.
There is a reason that both the frequentist and the Bayesian approaches exist.
You can calculate wrong in a way that overestimates the probability, even if the probability you estimate is small. For some highly improbable events, if you calculate a probability of 10^-9 your best estimate of the probability might be smaller than that.
True. I suppose I was unconsciously thinking (now there’s a phrase to fear!) about improbable dangerous events, where it is much more important not to underestimate P(X). If I get it wrong such that P(X) is truly only one in a trillion, then I am never going to know the difference and it’s not a big deal, but if P(X) is truly on the order of P(I suck at maths) then I am in serious trouble ;)
Especially given the recent evidence you have just provided for that hypothesis.
I’ve never been completely happy with the “I could make 1M similar statements and be wrong once” test. It seems, I dunno, kind of a frequentist way of thinking about the probability that I’m wrong. I can’t imagine making a million statements and have no way of knowing what it’s like to feel confidence about a statement to an accuracy of one part per million.
Other ways to think of tiny probabilities:
(1) If probability theory tells me there’s a 1 in a billion chance of X happening, then P(X) is somewhere between 1 in a billion and P(I calculated wrong), the latter being much higher.
If I were running on hardware that was better at arithmetic, P(I calculated wrong) could be got down way below 1 in a billion. After all, even today’s computers do billions of arithmetic operations per second. If they had anything like a one-in-a-billion failure rate per operation, we’d find them much less useful.
(2) Think of statements like P(7 is prime) = 1 as useful simplifications. If I am examining whether 7 is prime, I wouldn’t start with a prior of 1. But if I’m testing a hypothesis about something else and it depends on (among other things) whether 7 is prime, I wouldn’t assign P(7 is prime) some ridiculously specific just-under-1 probability; I’d call it 1 and simplify the causal network accordingly.
There are numerous studies that show that our brain’s natural way of thinking out probabilities is in terms of frequencies, and that people show less bias when presented with frequencies than when they are presented with percentages.
Thinking about which probabilities?
Probability is a complex concept. The probability in the sentence “the probability of getting more than 60 heads in 100 fair coin tosses” is a very different beast from the probability in the sentence “the probability of rain tomorrow”.
There is a reason that both the frequentist and the Bayesian approaches exist.
You can calculate wrong in a way that overestimates the probability, even if the probability you estimate is small. For some highly improbable events, if you calculate a probability of 10^-9 your best estimate of the probability might be smaller than that.
True. I suppose I was unconsciously thinking (now there’s a phrase to fear!) about improbable dangerous events, where it is much more important not to underestimate P(X). If I get it wrong such that P(X) is truly only one in a trillion, then I am never going to know the difference and it’s not a big deal, but if P(X) is truly on the order of P(I suck at maths) then I am in serious trouble ;)
Especially given the recent evidence you have just provided for that hypothesis.
T-t-t-the Ultimate Insult, aimed at… oh my… /me faints