is the correction to the probability density function really what you want, and are other deviations from Gaussianity expressible with cumulants? All I can think of is that the Gaussian is the maximum entropy distribution so maybe there’s a formula for how far below the maximum entropy you are. I don’t know what it’d be good for though.
I’m not exactly sure about “what you want”. It is not the case that you can exactly reconstruct most probability distributions you’ll encounter in real life from their moments/ cumulants (hence the expansion is perturbative, not exact).
But in the interpretability/ field-theoretic model of wide NN’s point of view, this is what you want (specifically, the fourth-order correction)
is the correction to the probability density function really what you want, and are other deviations from Gaussianity expressible with cumulants? All I can think of is that the Gaussian is the maximum entropy distribution so maybe there’s a formula for how far below the maximum entropy you are. I don’t know what it’d be good for though.
I’m not exactly sure about “what you want”. It is not the case that you can exactly reconstruct most probability distributions you’ll encounter in real life from their moments/ cumulants (hence the expansion is perturbative, not exact).
But in the interpretability/ field-theoretic model of wide NN’s point of view, this is what you want (specifically, the fourth-order correction)