Sierra Mike comments on Six (and a half) intuitions for KL divergence

[Sierra Mike]

This is nitpicky but I believe the reasoning for 3 is mistaken, mainly because of the counterintuitive claim that minimising cross-entropy over $\theta$ is equivalent to maximising KL divergence.

Writing the log-likelihood in the equation

$$\frac{1}{N}\log \ell =\frac{1}{N}\sum _{i=1}^N\ln Q_{\theta }\left(x_i\right),$$

we have the RHS tends to the negative cross entropy (by LLN), instead of just the cross entropy. Then, since $H\left(P\right)=H(P,Q)-D_{KL}\left(P||Q\right),$ with $P$ being the fixed (true) distribution and $Q$ varying with $\theta$, we have that MLE means we minimise the cross entropy which means we minimise the KL divergence.