abhatt349 comments on Re-Examining LayerNorm

[abhatt349]

Sorry for the mundane comment, but in the “Isolating the Nonlinearity” section of the colab notebook, you say

Note that a vector in $n$ dimensions with mean 0 has variance 1 if and only if it has length $\frac{1}{\sqrt{n}}$

I think you might’ve meant to say $\sqrt{n}$ there instead of $\frac{1}{\sqrt{n}}$, but please do correct me if I’m wrong!!!