is the correction to the probability density function really what you want, and are other deviations from Gaussianity expressible with cumulants? All I can think of is that the Gaussian is the maximum entropy distribution so maybe there’s a formula for how far below the maximum entropy you are. I don’t know what it’d be good for though.
I’m not exactly sure about what you mean wrt “what you want” here. It is not the case that you can exactly reconstruct most probability distributions you’ll encounter in real life from their moments/ cumulants (hence the expansion is perturbative, not exact).
But in the interpretability/ field-theoretic model of wide NN’s point of view, this is what you want (specifically, the fourth-order correction)
Well, usually I’m not inherently interested in a probability density function, I’m using it to calculate something else, like a moment or an entropy or something. But I guess I’ll see what you use it for in future posts.
is the correction to the probability density function really what you want, and are other deviations from Gaussianity expressible with cumulants? All I can think of is that the Gaussian is the maximum entropy distribution so maybe there’s a formula for how far below the maximum entropy you are. I don’t know what it’d be good for though.
I’m not exactly sure about what you mean wrt “what you want” here. It is not the case that you can exactly reconstruct most probability distributions you’ll encounter in real life from their moments/ cumulants (hence the expansion is perturbative, not exact).
But in the interpretability/ field-theoretic model of wide NN’s point of view, this is what you want (specifically, the fourth-order correction)
Well, usually I’m not inherently interested in a probability density function, I’m using it to calculate something else, like a moment or an entropy or something. But I guess I’ll see what you use it for in future posts.