So the oscillating phase formula is about approximately integrating the function exp(−f(x)/ℏ) against various “priors” p(x) (or more generally any fixed function g), where f is a Lagrangian (think energy) and (\hbar) is a small parameter. It gives an asymptotic series in powers of ℏ. The key point is that (more or less) the kth perturbative term only depends on the kth-order power series expansion of f around the “stationary points” (i.e., saddlepoints, Jac(f) = 0) when f is imaginary, on the maxima of f when f is real, and there is a mixed form that depends on stationary points of the imaginary part which are also maxima of the real part (if these exist); the formulae are all exactly the same, with the only difference between real and imaginary f (i.e. statistical vs. quantum mechanics) being whether you only keep maxima or all saddle points.
Now in SLT, you’re exactly applying the “real” stationary phase formula, i.e., looking at maxima of the (negative) loss function -L(w). The key thing that can happen is that there can be infinitely many maxima, and these might be singular (both in the sense of having higher degree of stationarity, and in the sense of forming a singular manifold). In this case the stationary phase formula is more complicated and AFAIK isn’t completely worked out; Watanabe was the first person who contributed to finding expressions for the general case here beyond the leading correction.
In the case of maxima which are nondegenerate, i.e., have positive-definite Hessian, the full perturbative expansion is known; in fact, at least in one very useful frame on it, terms are indexed by Feynman diagrams.
Now the energy function f that appears in this context is the log of the Fourier transform ^p(θ) of a probability distribution p(x). Notice that p(x) satisfies p(x)≥0 and ∫p(x)dx=1. This means that (\hat{p}(0) = \int f(x) dx) is 1 and its log is 0. You can check that all other values of the Fourier transform are ≤1 in absolute value (this follows from the fact that ∫|f(x)|≥|∫f(x)|). In fact, the Hessian is equal (up to scale) to the variance of p. Now the point is that the only way that variance can be zero is if your pd is concentrated on a lower-dimensional affine subspace, in which case you can simply reduce your problem to a lower-dimensional one with nonsingular Hessian. When this doesn’t happen, the function you’re applying stationary phase to has only nondegenerate maxima, and so the “standard” Feynman-diagram formula applies instead of the more sophisticated Watanabe one that’s used in SLT.
So the oscillating phase formula is about approximately integrating the function exp(−f(x)/ℏ) against various “priors” p(x) (or more generally any fixed function g), where f is a Lagrangian (think energy) and (\hbar) is a small parameter. It gives an asymptotic series in powers of ℏ. The key point is that (more or less) the kth perturbative term only depends on the kth-order power series expansion of f around the “stationary points” (i.e., saddlepoints, Jac(f) = 0) when f is imaginary, on the maxima of f when f is real, and there is a mixed form that depends on stationary points of the imaginary part which are also maxima of the real part (if these exist); the formulae are all exactly the same, with the only difference between real and imaginary f (i.e. statistical vs. quantum mechanics) being whether you only keep maxima or all saddle points.
Now in SLT, you’re exactly applying the “real” stationary phase formula, i.e., looking at maxima of the (negative) loss function -L(w). The key thing that can happen is that there can be infinitely many maxima, and these might be singular (both in the sense of having higher degree of stationarity, and in the sense of forming a singular manifold). In this case the stationary phase formula is more complicated and AFAIK isn’t completely worked out; Watanabe was the first person who contributed to finding expressions for the general case here beyond the leading correction.
In the case of maxima which are nondegenerate, i.e., have positive-definite Hessian, the full perturbative expansion is known; in fact, at least in one very useful frame on it, terms are indexed by Feynman diagrams.
Now the energy function f that appears in this context is the log of the Fourier transform ^p(θ) of a probability distribution p(x). Notice that p(x) satisfies p(x)≥0 and ∫p(x)dx=1. This means that (\hat{p}(0) = \int f(x) dx) is 1 and its log is 0. You can check that all other values of the Fourier transform are ≤1 in absolute value (this follows from the fact that ∫|f(x)|≥|∫f(x)|). In fact, the Hessian is equal (up to scale) to the variance of p. Now the point is that the only way that variance can be zero is if your pd is concentrated on a lower-dimensional affine subspace, in which case you can simply reduce your problem to a lower-dimensional one with nonsingular Hessian. When this doesn’t happen, the function you’re applying stationary phase to has only nondegenerate maxima, and so the “standard” Feynman-diagram formula applies instead of the more sophisticated Watanabe one that’s used in SLT.