johnswentworth comments on Generalizing Koopman-Pitman-Darmois

[johnswentworth]

The nice thing about using odds (or log odds) is that the normalizer cancels out when using Bayes’ rule. For a boolean query $X$ and data $D$, it looks like this:

• Bayes’ rule, usual form: $P\left[X|D\right]=\frac{1}{Z}P\left[D|X\right]P\left[X\right]$, where $Z$ is the normalizer

• Bayes’ rule, odds form: $\frac{P\left[X|D\right]}{P\left[\overline{X}|D\right]}=\frac{Z}{Z}\frac{P\left[D|X\right]}{P\left[D|\overline{X}\right]}\frac{P\left[X\right]}{P\left[\overline{X}\right]}=\frac{P\left[D|X\right]}{P\left[D|\overline{X}\right]}\frac{P\left[X\right]}{P\left[\overline{X}\right]}$, where $\overline{X}$ is not-$X$.

That’s the usual presentation. But note that it assumes $X$ is boolean. How do we get the same benefit—i.e. a form which in which the normalizer cancels out in Bayes’ rule—for non-boolean $X$?

The trick is to choose a reference value of $X$, and compute probabilities relative to the probability of that reference value. For instance, if $X$ is a six-sided die roll, I could choose $X=5$ as my reference value, and then I’d represent the distribution as $(x\mapsto \frac{P[X=x]}{P[X=5]})$. You can check that, when I update this distribution to $P\left[X|D\right]$, and represent the updated distribution as $(x\mapsto \frac{P[X=x|D]}{P[X=5|D]})$, the normalizer cancels out just like it does for the odds form on a boolean variable.

That’s the trick used for $\theta$ in the post. Applying the trick requires picking an arbitrary value of $\theta$ to use as the reference value (like $X=5$ above), and that’s ${\theta }^0$.