tailcalled comments on The Least Controversial Application of Geometric Rationality

[tailcalled]

Observation to potentially connect this to some math that people might be more familiar with: when $P$ and $Q$ are probability distributions, then $G_{x\sim P}\left[Q\right(x\left)\right]=e^{E_{x\sim P}\left[\ln Q\right(x\left)\right]}=e^{-H(P,Q)}$, where $H$ is the cross-entropy.

[Scott Garrabrant]

Note that the cross entropy, (and thus $G_{x\sim P}\left[Q\right(x\left)\right]$) is dependent on meaningless details of what events you consider the same vs different, but$e^{H(P,P)-H(P,Q)}=G_{x\sim P}\left[Q\right(x\left)\right]/G_{x\sim P}\left[P\right(x\left)\right]=G_{x\sim P}\left[Q\right(x\left)/P\right(x\left)\right]$ is not (as much), and when maximizing with respect to $Q$, this is the same maximization.

(I am just pointing out that KL divergence is a more natural concept than cross entropy.)

[tailcalled]

The middle piece here should be $G_{x\sim P}\left[Q\right(x\left)\right]/G_{x\sim P}\left[P\right(x\left)\right]$, right?

Anyway KL-divergence is based.

[Scott Garrabrant]

Yeah, edited.

[tailcalled]

I think $e^{-H\left(P\right)}$ might also have a natural interpretation of something along the lines of “The probability that two consecutive samples from P are equal”. This holds exactly for the uniform distribution, but only holds approximately for the Bernoulli distribution, so this is not a perfect heuristic.