It would be nice to have a couple examples comparing concrete distributions Q and P and examining their KL-divergence, why it’s large or small, and why it’s not symmetric.
I think some of the responses here do a pretty good job of this. It’s not really what I intended to go into with my post since I was trying to keep it brief (although I agree this seems like it would be useful).
It would be nice to have a couple examples comparing concrete distributions Q and P and examining their KL-divergence, why it’s large or small, and why it’s not symmetric.
I think some of the responses here do a pretty good job of this. It’s not really what I intended to go into with my post since I was trying to keep it brief (although I agree this seems like it would be useful).