First off, it seems weird to say “I have 33 bits of evidence that this bit is a 1.”
It seems weird to me because the bits of “33 bits” looks like the same units as the bit of “this bit”, but they aren’t the same. Map/territory. From now on, I’m calling the first, A-bits, and the second, B-bits.
Why does it take an infinite number of bits of evidence to get 1 bit of information?
It takes an infinite number of A-bits to know with absolute certainty one B-bit.
But that means a one-bit evidence-giver is someone who is right 2⁄3 of the time. Why the 2/3? That seems weird.
It seems weird to me because the bits of “33 bits” looks like the same units as the bit of “this bit”, but they aren’t the same. Map/territory. From now on, I’m calling the first, A-bits, and the second, B-bits.
It takes an infinite number of A-bits to know with absolute certainty one B-bit.
What were you expecting?