Clearly log-odds aren’t perfectly isomorphic with multiplicative probabilities, since clearly one allows probabilities of 0 and 1 and the other doesn’t.
Bayes’s theorem does assume nonzero probability, as you can observe by examining its proof.
Pr[A & B] = Pr[B] Pr[A|B] = Pr[A] Pr[B|A] by definition of conditional probability.
Pr[A|B] = Pr[A] Pr[B|A] / Pr[B] if we divide by Pr[B]. This assumes Pr[B]>0 because otherwise this operation is invalid.
You can’t derive properties of probability from Russell’s axioms, because these describe set theory and not probability. One standard way of deriving properties of probability is via Dutch Book arguments. These can only show that probabilities must be in the range [0,1] (including the endpoints). In fact, no finite sequence of bets you offer me can distinguish a credence of 1 from a credence of 1-epsilon for sufficiently small epsilon. (That is, for any epsilon, there’s a bet that distinguishes 1-epsilon from 1, but for any sequence of bets, there’s an 1-epsilon that is indistinguishable from 1).
Here is an analogy. The well-known formula D = RT describes the relationship between distance traveled, average speed, and time. You can also express this as log(D) = log(R) + log(T) if you like, or D/R = T. In either of these formulas, setting R=0 will be an error. This doesn’t mean that there’s no such thing as a speed of 0, and if you think your speed is 0 you are actually traveling at a speed of epsilon for some very small value of epsilon. It just means that when you passed to these (mostly equivalent) formulations, you lost the capability to discuss speeds of 0. In fact, when we set R to 0 in the original formula, we get a more useful description of what happens: D=0 no matter the value of T. In other words, 0 is a valid speed, but you can’t travel a nonzero distance with an average speed of zero, no matter how much time you allow yourself.
What is the difference between log-odds and log-speeds, that makes the former an isomorphism and the latter an imperfect description?
Finally, do you really think that someone who thinks “0 and 1 are probabilities” is a statement LW is irrational about is unaware of the “0 and 1 are not probabilities” post?
By virtue of the definition of a logarithm, exp(log(x)) = x, we can derive that since the exponential function is well-defined for complex numbers, so is the logarithm. Taking the logarithm of a negative number nets you the logarithm of the absolute plus imaginary pi. The real part of any logarithm is a symmetric function, and there are probably a few other interesting properties of logarithms in complex analysis that I don’t know of.
log(0) is undefined, as you note, but that does not mean the limit x → 0 log(x) is undefined. It is in fact a pole singularity (if I have understood my singularity analysis correct). No matter how you approach it, you get negative infinity. So given your “logarithmic velocities,” I counter with the fact that using a limit it is still a perfect isomorphism. Limit of x → -inf exp(x) is indeed 0, so when using limits (which is practically what Real Numbers are for) your argument that log isn’t an isomophism from reals to reals is hereby proven invalid (if you want a step by step proof, just ask, it’ll be a fun exercise).
Given that logarithms are a category-theoretic isomorphism from the reals onto the reals (from the multiplication group onto the addition group) there is no reason why log-odds isn’t as valid as odds, which is as valid as ]0,1[ probabilities. Infinities are not valid Reals, 0 and 1 are not valid probablities QED.
As I said. Do not challenge anyone (including me) on the abstract algebra of the matter.
[I do apologize if the argument is poorly formulated, I am as of writing mildly intoxicated]
Yes, log(x) is an isomorphism from the positive reals as a multiplicative group to the real numbers as an additive group. As a result, it is only an isomorphism from multiplicative probabilities to additive log-probabilities if you assume that 0 is not a probability to begin with, which is circular logic.
To obtain paradoxes, it is you that would need access to more proofs than I do.
From an evidence-based point of view, as a contrapositive of the usual argument against P=0 and P=1, we can say that if it’s possible to convince me that a statement might be false, it must be that I already assign it probability strictly <1.
As you may have guessed, I also don’t agree with the point of view that I can be convinced of the truth of any statement, even given arbitrarily bizarre circumstances. I believe that one needs rules by which to reason. Obviously these can be changed, but you need meta-rules to describe how you change rules, and possibly meta-meta-rules as well, but there must be something basic to use.
So I assign P=1 to things that are fundamental to the way I think about the world. In my case, this includes the way I think about probability.
Clearly log-odds aren’t perfectly isomorphic with multiplicative probabilities, since clearly one allows probabilities of 0 and 1 and the other doesn’t.
Bayes’s theorem does assume nonzero probability, as you can observe by examining its proof.
Pr[A & B] = Pr[B] Pr[A|B] = Pr[A] Pr[B|A] by definition of conditional probability.
Pr[A|B] = Pr[A] Pr[B|A] / Pr[B] if we divide by Pr[B]. This assumes Pr[B]>0 because otherwise this operation is invalid.
You can’t derive properties of probability from Russell’s axioms, because these describe set theory and not probability. One standard way of deriving properties of probability is via Dutch Book arguments. These can only show that probabilities must be in the range [0,1] (including the endpoints). In fact, no finite sequence of bets you offer me can distinguish a credence of 1 from a credence of 1-epsilon for sufficiently small epsilon. (That is, for any epsilon, there’s a bet that distinguishes 1-epsilon from 1, but for any sequence of bets, there’s an 1-epsilon that is indistinguishable from 1).
Here is an analogy. The well-known formula D = RT describes the relationship between distance traveled, average speed, and time. You can also express this as log(D) = log(R) + log(T) if you like, or D/R = T. In either of these formulas, setting R=0 will be an error. This doesn’t mean that there’s no such thing as a speed of 0, and if you think your speed is 0 you are actually traveling at a speed of epsilon for some very small value of epsilon. It just means that when you passed to these (mostly equivalent) formulations, you lost the capability to discuss speeds of 0. In fact, when we set R to 0 in the original formula, we get a more useful description of what happens: D=0 no matter the value of T. In other words, 0 is a valid speed, but you can’t travel a nonzero distance with an average speed of zero, no matter how much time you allow yourself.
What is the difference between log-odds and log-speeds, that makes the former an isomorphism and the latter an imperfect description?
Finally, do you really think that someone who thinks “0 and 1 are probabilities” is a statement LW is irrational about is unaware of the “0 and 1 are not probabilities” post?
Potholing that last sentence was mostly for fun.
By virtue of the definition of a logarithm, exp(log(x)) = x, we can derive that since the exponential function is well-defined for complex numbers, so is the logarithm. Taking the logarithm of a negative number nets you the logarithm of the absolute plus imaginary pi. The real part of any logarithm is a symmetric function, and there are probably a few other interesting properties of logarithms in complex analysis that I don’t know of.
log(0) is undefined, as you note, but that does not mean the limit x → 0 log(x) is undefined. It is in fact a pole singularity (if I have understood my singularity analysis correct). No matter how you approach it, you get negative infinity. So given your “logarithmic velocities,” I counter with the fact that using a limit it is still a perfect isomorphism. Limit of x → -inf exp(x) is indeed 0, so when using limits (which is practically what Real Numbers are for) your argument that log isn’t an isomophism from reals to reals is hereby proven invalid (if you want a step by step proof, just ask, it’ll be a fun exercise).
Given that logarithms are a category-theoretic isomorphism from the reals onto the reals (from the multiplication group onto the addition group) there is no reason why log-odds isn’t as valid as odds, which is as valid as ]0,1[ probabilities. Infinities are not valid Reals, 0 and 1 are not valid probablities QED.
As I said. Do not challenge anyone (including me) on the abstract algebra of the matter.
[I do apologize if the argument is poorly formulated, I am as of writing mildly intoxicated]
Yes, log(x) is an isomorphism from the positive reals as a multiplicative group to the real numbers as an additive group. As a result, it is only an isomorphism from multiplicative probabilities to additive log-probabilities if you assume that 0 is not a probability to begin with, which is circular logic.
So, pray tell: When are P=0 and P=1 applicable? Don’t you get paradoxes? What prior allows you to attain them?
I am really genuinely curious what sort of proofs you have access to that I do not.
To obtain paradoxes, it is you that would need access to more proofs than I do.
From an evidence-based point of view, as a contrapositive of the usual argument against P=0 and P=1, we can say that if it’s possible to convince me that a statement might be false, it must be that I already assign it probability strictly <1.
As you may have guessed, I also don’t agree with the point of view that I can be convinced of the truth of any statement, even given arbitrarily bizarre circumstances. I believe that one needs rules by which to reason. Obviously these can be changed, but you need meta-rules to describe how you change rules, and possibly meta-meta-rules as well, but there must be something basic to use.
So I assign P=1 to things that are fundamental to the way I think about the world. In my case, this includes the way I think about probability.