You’re basically right, in that it requires much stronger evidence to move you from 45% → 55% credence than to move you from 90% → 99.9%.
It is helpful to think in terms of likelihood ratios. To go from 90% → 99.9% credence requires observing evidence with a likelihood ratio of (0.999 / 0.001) / (0.9 / 0.1) = 111, which is about 6.8 bits of evidence. [edit: fixed flipped / wrong math]
To go from 45% → 55% credence, you just need a likelihood ratio of (0.55/.45)/(0.45/0.55) = 1.5, or about 0.6 bits of evidence.
(Getting to 100% credence via Bayesian updating requires +inf bits of evidence; remember that 100% isn’t really a probability.)
Re getting to 100% probability of a outcome, that’s actually surprisingly easy to do sometimes, especially in infinite sets like the real numbers. It’s not trivial, but you can get these outcomes sometimes.
Sure, you can also easily imagine a set with measure 1 or 0. My remark was just a reminder that 1 and 0 are not probabilities, in the same sense that infinity is not a real number.
Just like you can get away with manipulating infinities intuitively if you’re careful, instead of treating everything formally via limits, you can also usually get away with treating 0 and 1 as ordinary probabilities. But you have to be careful, and it makes the math in cases like the OP less intuitive than just using likelihood ratios and logarithms, IMO.
My remark was just a reminder that 1 and 0 are not probabilities, in the same sense that infinity is not a real number.
This is admittedly a minor internet crusade/pet peeve of mine, but the claim that 1 and 0 aren’t probabilities is exactly wrong, and the analogy is pretty strained here. In fact, probability theory need to have 0 and 1 as legitimate probabilities, or things fall apart into incoherency.
The post you linked is one of the most egregiously wrong things Eliezer has ever said that is purely mathematical.
And we don’t just imagine measure/probability 1 or 0 sets, we have proved that certain sets of this kind exists.
There are fundamental disanalogies that make infinity not a number (except in the extended real line and the projective real line), compared to 0 and 1 being probabilities.
If you want to prove a bunch of theorems involving continuities and infinities, treating 0 and 1 as probabilities is much more elegant and things mostly fall apart without them, yes.
If your goal is to reason under uncertainty, thinking in terms of odds ratios and decibels is a way of putting your map in close correspondence with the territory. Allowing for infinities in this use case introduces complications and weird philosophical questions about the (in)finiteness of reality.
On earth, most people start out by learning probability theory in terms of probabilities, for the purpose of solving math problems or proving theorems in school. Later (if they stumble across the right kinds of blogs) they learn probability as a reasoning tool, but often forget or don’t realize that thinking in terms of odds ratios when using probability for this purpose is much more convenient once you get used to it.
On a planet where people grew up studying probability as a reasoning tool first, and only as an afterthought studied it as a branch of math, someone might need to write a blog post that 0 and 1 are basically just ordinary probabilities, and sometimes probabilities are more elegant and intuitive than odds and decibels, lest people start over-complicating their proofs.
I don’t see anything wrong or contradictory with pointing out the difference between probability as mathematical theory and probability as reasoning method.
You’re basically right, in that it requires much stronger evidence to move you from 45% → 55% credence than to move you from 90% → 99.9%.
It is helpful to think in terms of likelihood ratios. To go from 90% → 99.9% credence requires observing evidence with a likelihood ratio of (0.999 / 0.001) / (0.9 / 0.1) = 111, which is about 6.8 bits of evidence. [edit: fixed flipped / wrong math]
To go from 45% → 55% credence, you just need a likelihood ratio of (0.55/.45)/(0.45/0.55) = 1.5, or about 0.6 bits of evidence.
(Getting to 100% credence via Bayesian updating requires +inf bits of evidence; remember that 100% isn’t really a probability.)
Re getting to 100% probability of a outcome, that’s actually surprisingly easy to do sometimes, especially in infinite sets like the real numbers. It’s not trivial, but you can get these outcomes sometimes.
Sure, you can also easily imagine a set with measure 1 or 0. My remark was just a reminder that 1 and 0 are not probabilities, in the same sense that infinity is not a real number.
Just like you can get away with manipulating infinities intuitively if you’re careful, instead of treating everything formally via limits, you can also usually get away with treating 0 and 1 as ordinary probabilities. But you have to be careful, and it makes the math in cases like the OP less intuitive than just using likelihood ratios and logarithms, IMO.
This is admittedly a minor internet crusade/pet peeve of mine, but the claim that 1 and 0 aren’t probabilities is exactly wrong, and the analogy is pretty strained here. In fact, probability theory need to have 0 and 1 as legitimate probabilities, or things fall apart into incoherency.
The post you linked is one of the most egregiously wrong things Eliezer has ever said that is purely mathematical.
And we don’t just imagine measure/probability 1 or 0 sets, we have proved that certain sets of this kind exists.
There are fundamental disanalogies that make infinity not a number (except in the extended real line and the projective real line), compared to 0 and 1 being probabilities.
If you want to prove a bunch of theorems involving continuities and infinities, treating 0 and 1 as probabilities is much more elegant and things mostly fall apart without them, yes.
If your goal is to reason under uncertainty, thinking in terms of odds ratios and decibels is a way of putting your map in close correspondence with the territory. Allowing for infinities in this use case introduces complications and weird philosophical questions about the (in)finiteness of reality.
On earth, most people start out by learning probability theory in terms of probabilities, for the purpose of solving math problems or proving theorems in school. Later (if they stumble across the right kinds of blogs) they learn probability as a reasoning tool, but often forget or don’t realize that thinking in terms of odds ratios when using probability for this purpose is much more convenient once you get used to it.
On a planet where people grew up studying probability as a reasoning tool first, and only as an afterthought studied it as a branch of math, someone might need to write a blog post that 0 and 1 are basically just ordinary probabilities, and sometimes probabilities are more elegant and intuitive than odds and decibels, lest people start over-complicating their proofs.
I don’t see anything wrong or contradictory with pointing out the difference between probability as mathematical theory and probability as reasoning method.