Scott Garrabrant comments on The Least Controversial Application of Geometric Rationality

[Scott Garrabrant]

Note that the cross entropy, (and thus $G_{x\sim P}\left[Q\right(x\left)\right]$) is dependent on meaningless details of what events you consider the same vs different, but$e^{H(P,P)-H(P,Q)}=G_{x\sim P}\left[Q\right(x\left)\right]/G_{x\sim P}\left[P\right(x\left)\right]=G_{x\sim P}\left[Q\right(x\left)/P\right(x\left)\right]$ is not (as much), and when maximizing with respect to $Q$, this is the same maximization.

(I am just pointing out that KL divergence is a more natural concept than cross entropy.)

[tailcalled]

The middle piece here should be $G_{x\sim P}\left[Q\right(x\left)\right]/G_{x\sim P}\left[P\right(x\left)\right]$, right?

Anyway KL-divergence is based.

[Scott Garrabrant]

Yeah, edited.