Charlie Steiner comments on Performing an SVD on a time-series matrix of gradient updates on an MNIST network produces 92.5 singular values

[Charlie Steiner]

I googled around for what you’d expect from the SVD of random matrices, turns out to be a thing. Probably a better null hypothesis.