romeostevensit comments on What happens to variance as neural network training is scaled? What does it imply about “lottery tickets”?

[romeostevensit]

One related question is what sub-tasks of gpt-3 showed surprise jackpots vs gpt-2