We all saw the GPT performance scaling graphs in the papers, and we all stared at them and imagined extending the trend for another five OOMs or so… but then Lanrian went and actually did it! Answered the question we had all been asking! And rigorously dealt with some technical complications along the way.
I’ve since referred to this post a bunch of times. It’s my go-to reference when discussing performance scaling trends.
We all saw the GPT performance scaling graphs in the papers, and we all stared at them and imagined extending the trend for another five OOMs or so… but then Lanrian went and actually did it! Answered the question we had all been asking! And rigorously dealt with some technical complications along the way.
I’ve since referred to this post a bunch of times. It’s my go-to reference when discussing performance scaling trends.