I’ve been thinking a lot about the “missing thing”. In fact I have some experiments planned, if I ever have the time and compute, to get at least an intuition about system 2 thinking in transformers.
But if you look at the PaLM-paper (and more generally at gwern’s collection of internal monologue examples) it sure looks like deliberate reasoning emerges in very large models.
If there is a “missing thing” I think it is more likely to be something about the representations learned by humans being right off the bat more “gears-level”. Maybe like Hawkin’s reference frames. Some decomposability that enables much more powerful abstraction and that has to be pounded into a NN with millions of examples.
That kind of “missing thing” would impact extrapolation, one-shot-learning, robust system 2 thinking, abstraction, long term planning, causal reasoning, thinking long on hard problems etc.
the representations learned by humans being right off the bat more “gears-level”. Maybe like Hawkin’s reference frames. Some decomposability that enables much more powerful abstraction and that has to be pounded into a NN with millions of examples.
That makes a lot of sense, and if it’s that’s true then that’s hopeful for interpretability efforts. It would be easier to read inside an ML model if it’s composed of parts that map to real concepts.
I’ve been thinking a lot about the “missing thing”. In fact I have some experiments planned, if I ever have the time and compute, to get at least an intuition about system 2 thinking in transformers.
But if you look at the PaLM-paper (and more generally at gwern’s collection of internal monologue examples) it sure looks like deliberate reasoning emerges in very large models.
If there is a “missing thing” I think it is more likely to be something about the representations learned by humans being right off the bat more “gears-level”. Maybe like Hawkin’s reference frames. Some decomposability that enables much more powerful abstraction and that has to be pounded into a NN with millions of examples.
That kind of “missing thing” would impact extrapolation, one-shot-learning, robust system 2 thinking, abstraction, long term planning, causal reasoning, thinking long on hard problems etc.
That makes a lot of sense, and if it’s that’s true then that’s hopeful for interpretability efforts. It would be easier to read inside an ML model if it’s composed of parts that map to real concepts.