I was curious to re-read the chat log, and had to do some digging on archive.org to find it. The guy made 17 bets about numbers being prime, and lost the bet on the 17th bet.
If it’s not clear why this doesn’t follow consider the anecdote Eliezer references in the quote above, which runs as follows: A gets B to agree that if 7 is not prime, B will give A $100. B then makes the same agreement for 11, 13, 17, 19, and 23. Then A asks about 27. B refuses. What about 29? Sure. 31? Yes. 33? No. 37? Yes. 39? No. 41? Yes. 43? Yes. 47? Yes. 49? No. 51? Yes. And suddenly B is $100 poorer.
Now, B claimed to be 100% sure about 7 being prime, which I don’t agree with. But that’s not what lost him his $100. What lost him his $100 is that, as the game went on, he got careless. If he’d taken the time to ask himself, “am I really as sure about 51 as I am about 7?” he’d probably have realized the answer was “no.” He probably didn’t check he primality of 51 as carefully as I checked the primality of 53 at the beginning of this post. (From the provided chat transcript, sleep deprivation may have also had something to do with it.)
If you tried to make 10,000 statements with 99.99% certainty, sooner or later you would get careless. Heck, before I started writing this post, I tried typing up a list of statements I was sure of, and it wasn’t long before I’d typed 1 + 0 = 10 (I’d meant to type 1 + 9 = 10. Oops.) But the fact that, as the exercise went on, you’d start including statements that weren’t really as certain as the first statement doesn’t mean you couldn’t be justified in being 99.99% certain of that first statement.
I do think this is an important counterpoint, but still, while I agree that if a person actually thought carefully about each prime number, they’d have made it much farther than a 1-out-of-17 failure rate, I’d still bet against them successfully making 10,000 careful statements without ever screwing up in some dumb way.
Anecdata: In the mobile game Golf Rivals, it is trivial to sink a putt from any distance on the green, with a little bit of care. I (and opponents) miss about 1 in 1000 times
I was curious to re-read the chat log, and had to do some digging on archive.org to find it. The guy made 17 bets about numbers being prime, and lost the bet on the 17th bet.
Transcript here
Sequence article that referenced it here.
Interesting followup by Chris Halliquist here:
I do think this is an important counterpoint, but still, while I agree that if a person actually thought carefully about each prime number, they’d have made it much farther than a 1-out-of-17 failure rate, I’d still bet against them successfully making 10,000 careful statements without ever screwing up in some dumb way.
Anecdata: In the mobile game Golf Rivals, it is trivial to sink a putt from any distance on the green, with a little bit of care. I (and opponents) miss about 1 in 1000 times