Current LLMs can only do sequential reasoning of any kind by adjusting their activations, not their weights, and this is probably not enough to derive and internalize new concepts à la C.
For me this is the key bit which makes me update towards your thesis.
I think this inability of “learning while thinking” might be the key missing thing of LLMs and I am not sure “thought assessment” or “sequential reasoning” are not red herrings compared to this. What good is assessment of thoughts if you are fundamentally limited in changing them? Also, reasoning models seem to do sequential reasoning just fine as long as they already have learned all the necessary concepts.
For me this is the key bit which makes me update towards your thesis.
I think this inability of “learning while thinking” might be the key missing thing of LLMs and I am not sure “thought assessment” or “sequential reasoning” are not red herrings compared to this. What good is assessment of thoughts if you are fundamentally limited in changing them? Also, reasoning models seem to do sequential reasoning just fine as long as they already have learned all the necessary concepts.