johnswentworth comments on Natural Latents: The Math

[johnswentworth]

Consider the exact version of the redundancy condition for latent $\Lambda$ over $X_1,X_2$:

$P[\Lambda ,X_1,X_2]=P\left[\Lambda |X_1\right]P[X_1,X_2]$

and

$P[\Lambda ,X_1,X_2]=P\left[\Lambda |X_2\right]P[X_1,X_2]$

Combine these two and we get, for all $\Lambda ,X_1,X_2$:

$P\left[\Lambda |X_1\right]=P\left[\Lambda |X_2\right]$ OR $P[X_1,X_2]=0$

That’s the foundation for a conceptually-simple method for finding the exact natural latent (if one exists) given a distribution $P[X_1,X_2]$:

• Pick a value $X_1,X_2$ which has nonzero probability, and initialize a set $S$ containing that value. Then we must have $P[\Lambda |X\in S]=P\left[\Lambda |X_1\right]=P\left[\Lambda |X_2\right]$ for all $\Lambda$.

• Loop: add to $S$ a new value $X_1^{\prime },X_2$ or $X_1,X_2^{\prime }$ where the value $X_2$ or $X_1$ (respectively) already appears in one of the pairs in $S$. Then $P\left[\Lambda |X_1^{\prime }\right]=P[\Lambda |X\in S]$ or $P\left[\Lambda |X_2^{\prime }\right]=P[\Lambda |X\in S]$, respectively. Repeat until there are no more candidate values to add to $S$.

• Pick a new pair and repeat with a new set, until all values of $X$ have been added to a set.

• Now take the latent to be the equivalence class in which $X$ falls.

Does that make sense?

[Lorxus]

As I also said in person, very much so!