Yup, that’s basically what I think! IMO, grokking = having memorised the “underlying rules” that define the DGP, and these rules are general by definition.”Reasoning” is a loaded term that’s difficult to unpack, but I think a good working definition is “applying a set of rules to arrive at an answer”. In other words, reasoning is learning a “correct algorithm” to solve the problem. Therefore being able to reason correctly 100% of the time is equivalent to models having grokked their problem domain.
See this work, which finds that reasoning only happens through grokking. Separate work has trained models to do tree search, and found that backwards chaining circuits (a correct algorithm) emerge only through grokking. And also the seminal work on modular addition which found that correct algorithms emerge through grokking.
Note that the question of “is reasoning in natural language grokkable?” is a totally separate crux and one which I’m highly uncertain about.
Yup, that’s basically what I think! IMO, grokking = having memorised the “underlying rules” that define the DGP, and these rules are general by definition.”Reasoning” is a loaded term that’s difficult to unpack, but I think a good working definition is “applying a set of rules to arrive at an answer”. In other words, reasoning is learning a “correct algorithm” to solve the problem. Therefore being able to reason correctly 100% of the time is equivalent to models having grokked their problem domain.
See this work, which finds that reasoning only happens through grokking. Separate work has trained models to do tree search, and found that backwards chaining circuits (a correct algorithm) emerge only through grokking. And also the seminal work on modular addition which found that correct algorithms emerge through grokking.
Note that the question of “is reasoning in natural language grokkable?” is a totally separate crux and one which I’m highly uncertain about.