There are really two claims here. The first one—that if some guy on the Internet has a model predicting X with 99.99% certainty, then you should assign less probability to X, absent other evidence—seems interesting, but relatively easy to accept. I’m pretty sure I’ve been reasoning this way in the past.
The second claim is exactly the same, but applied to oneself. “If I have come up with an argument that predicts X with 99.99% certainty, I should be less than 99.99% certain of X.” This is not something that people do by default. I doubt that I do it unless prompted. Great post!
Stylistic nitpick, though: things like “999,999,999 in a billion” are tricky to parse, especially when compared to “999,999,999,999 in a trillion” (which I initially read as approximately 1 in 1000 before counting the 9s) or “1/1 billion”. Counting the 9s is part of the problem, the other that the numerator is a number and the denominator is a word. What’s wrong with writing 99.99% and 99.9999%? These are different from the original values in the post, but still carry the argument, and are easier to read.
There are really two claims here. The first one—that if some guy on the Internet has a model predicting X with 99.99% certainty, then you should assign less probability to X, absent other evidence—seems interesting, but relatively easy to accept. I’m pretty sure I’ve been reasoning this way in the past.
The second claim is exactly the same, but applied to oneself. “If I have come up with an argument that predicts X with 99.99% certainty, I should be less than 99.99% certain of X.” This is not something that people do by default. I doubt that I do it unless prompted. Great post!
Stylistic nitpick, though: things like “999,999,999 in a billion” are tricky to parse, especially when compared to “999,999,999,999 in a trillion” (which I initially read as approximately 1 in 1000 before counting the 9s) or “1/1 billion”. Counting the 9s is part of the problem, the other that the numerator is a number and the denominator is a word. What’s wrong with writing 99.99% and 99.9999%? These are different from the original values in the post, but still carry the argument, and are easier to read.
I personally find the best way to deal with such numbers is to talk about nines.
999,999,999 in a billion=99.9 999 999%= 9 nines
999,999,999,999 in a trillion=99.9 999 999 999%= 12 nines