Yeah, that seems right, I think I’m basically at “no, you can’t just 10x the learning rate once grokking starts”.
Yeah, that seems right, I think I’m basically at “no, you can’t just 10x the learning rate once grokking starts”.