interstice comments on Degeneracies are sticky for SGD

[interstice]

Possibly related: Stochastic Collapse: How Gradient Noise Attracts SGD towards sparser subnetworks”