The paper “Auto-Regressive Next-Token Predictors are Universal Learners” made me a little more skeptical of attributing general reasoning ability to LLMs. They show that even linear predictive models, basically just linear regression, can technically perform any algorithm when used autoregressively like with chain-of-thought. The results aren’t that mind-blowing but it made me wonder whether performing certain algorithms correctly with a scratchpad is as much evidence of intelligence as I thought.
One man’s modus ponens is another man’s modus tollens, and what I do take away from the result is that intelligence with enough compute is too easy to do, so easy that even linear predictive models can do it in theory.
So they don’t disprove that intelligent/algorithmic reasoning isn’t happening in LLMs, but rather that it’s too easy to get intelligence/computation by many different methods.
It’s similar to the proof that an origami computer can compute every function computable by a Turing Machine, and if in a hypothetical world we were instead using very large origami pieces to build up AIs like AlphaGo, I don’t think that there would be a sense in which it’s obviously not reasoning about the game of Go.
I agree that origami AIs would still be intelligent if implementing the same computations. I was trying to point at LLMs potentially being ‘sphexish’: having behaviors made of baked if-then patterns linked together that superficially resemble ones designed on-the-fly for a purpose. I think this is related to what the “heuristic hypothesis” is getting at.
IMO, I think the heuristic hypothesis is partially right, but partially right is the keyword, in the sense that LLMs both will have sphexish heuristics and mostly clean algorithms for solving problems.
I also expect OpenAI to broadly move LLMs from more heuristic-like reasoning to algorithmic-like reasoning, and o1 is slight evidence towards more systematic reasoning in LLMs.
The paper “Auto-Regressive Next-Token Predictors are Universal Learners” made me a little more skeptical of attributing general reasoning ability to LLMs. They show that even linear predictive models, basically just linear regression, can technically perform any algorithm when used autoregressively like with chain-of-thought. The results aren’t that mind-blowing but it made me wonder whether performing certain algorithms correctly with a scratchpad is as much evidence of intelligence as I thought.
One man’s modus ponens is another man’s modus tollens, and what I do take away from the result is that intelligence with enough compute is too easy to do, so easy that even linear predictive models can do it in theory.
So they don’t disprove that intelligent/algorithmic reasoning isn’t happening in LLMs, but rather that it’s too easy to get intelligence/computation by many different methods.
It’s similar to the proof that an origami computer can compute every function computable by a Turing Machine, and if in a hypothetical world we were instead using very large origami pieces to build up AIs like AlphaGo, I don’t think that there would be a sense in which it’s obviously not reasoning about the game of Go.
https://www.quantamagazine.org/how-to-build-an-origami-computer-20240130/
I agree that origami AIs would still be intelligent if implementing the same computations. I was trying to point at LLMs potentially being ‘sphexish’: having behaviors made of baked if-then patterns linked together that superficially resemble ones designed on-the-fly for a purpose. I think this is related to what the “heuristic hypothesis” is getting at.
IMO, I think the heuristic hypothesis is partially right, but partially right is the keyword, in the sense that LLMs both will have sphexish heuristics and mostly clean algorithms for solving problems.
I also expect OpenAI to broadly move LLMs from more heuristic-like reasoning to algorithmic-like reasoning, and o1 is slight evidence towards more systematic reasoning in LLMs.