Yes, so an example of this would be the ReLU scaling symmetry discussed in “Neural networks are freaks of symmetries.” You’re right that regularization often breaks this kind of symmetry.
But even when there are no local symmetries, having other points that have the same posterior means this assumption of asymptotic normality doesn’t hold.
Yes, so an example of this would be the ReLU scaling symmetry discussed in “Neural networks are freaks of symmetries.” You’re right that regularization often breaks this kind of symmetry.
But even when there are no local symmetries, having other points that have the same posterior means this assumption of asymptotic normality doesn’t hold.