Yeah, I agree that a lot of the “phase transitions” look more discontinuous than they actually are due to the log on the x axis — the OG grokking paper definitely commits this sin, for example.
(I think there’s also another disagreement here about how close humans are to this natural limit.)
Yeah, I agree that a lot of the “phase transitions” look more discontinuous than they actually are due to the log on the x axis — the OG grokking paper definitely commits this sin, for example.
(I think there’s also another disagreement here about how close humans are to this natural limit.)