It definitely feels like there is still something missing, something that these generative models lack no matter how impressive they get. Most people agree that the GPT-likes don’t seem to have the right type specification to trigger the utopic singularity/horrible doom of humanity.
Maybe it’s agency. Reinforcement learning still lags behind visual and language models. We still could not train a robot to do most of the things a you could train a monkey to do, even as we have systems that appear to speak like humans.
A couple more candidates for “the missing thing”: maybe code generation (Copilot N.0) will be able to something really impressive/scary. But I feel like that’s just having AI solve the problem for us, because the logical thing to do would be to have Copilot N.0 program a real AGI.
The thing that I’m watching closely is using feedback loops on models designed for multistep reasoning, which might be like Kaneman’s System 2. Many have noted that deep learning models, especially generative and discriminative models, resemble System 1. A reasoning feedback loop matches my intuition about how my own brain works (when I’m actually thinking and not in some other brain state like enjoying the moment).
Question for everyone: Do you feel like there is one “missing thing” between these generative models and AGI, and what is it? Or do you think these generative models are not on the path to AGI, however impressive they are?
I’ve been thinking a lot about the “missing thing”. In fact I have some experiments planned, if I ever have the time and compute, to get at least an intuition about system 2 thinking in transformers.
But if you look at the PaLM-paper (and more generally at gwern’s collection of internal monologue examples) it sure looks like deliberate reasoning emerges in very large models.
If there is a “missing thing” I think it is more likely to be something about the representations learned by humans being right off the bat more “gears-level”. Maybe like Hawkin’s reference frames. Some decomposability that enables much more powerful abstraction and that has to be pounded into a NN with millions of examples.
That kind of “missing thing” would impact extrapolation, one-shot-learning, robust system 2 thinking, abstraction, long term planning, causal reasoning, thinking long on hard problems etc.
the representations learned by humans being right off the bat more “gears-level”. Maybe like Hawkin’s reference frames. Some decomposability that enables much more powerful abstraction and that has to be pounded into a NN with millions of examples.
That makes a lot of sense, and if it’s that’s true then that’s hopeful for interpretability efforts. It would be easier to read inside an ML model if it’s composed of parts that map to real concepts.
It’s pretty difficult to tell intuitively because the human mind is programmed to anthropomorphize. It’s a binary recognition; either it looks 100% human or it doesn’t.
We’re not built to compare 2 different systems that can do some human subroutines but not others. So AI could make a big leap approaching general intelligence, and the lion’s share of that leap could be visible or invisible based on how much the resulting behavior reminds us of ourselves.
Due to the anthropic principle, general intelligence could have a one-in-an-octillion chance of ever randomly evolving, anywhere, ever, and we would still be here observing all the successful steps having happened, because if all the steps didn’t happen then we wouldn’t be here observing anything. There would still be tons of animals like ants and chimpanzees because evolution always creates a ton of alternative “failed” offshoots. So it’s always possible that there’s some logical process that’s necessary for general intelligence, and we’re astronomically unlikely to discover it randomly, through brute forcing or even innovation, until we pinpoint all the exact lines of code in the human brain that distinguishes our intelligence from chimpanzees. But that’s only a possibility, far from a guarantee.
It definitely feels like there is still something missing, something that these generative models lack no matter how impressive they get. Most people agree that the GPT-likes don’t seem to have the right type specification to trigger the utopic singularity/horrible doom of humanity.
Maybe it’s agency. Reinforcement learning still lags behind visual and language models. We still could not train a robot to do most of the things a you could train a monkey to do, even as we have systems that appear to speak like humans.
A couple more candidates for “the missing thing”: maybe code generation (Copilot N.0) will be able to something really impressive/scary. But I feel like that’s just having AI solve the problem for us, because the logical thing to do would be to have Copilot N.0 program a real AGI.
The thing that I’m watching closely is using feedback loops on models designed for multistep reasoning, which might be like Kaneman’s System 2. Many have noted that deep learning models, especially generative and discriminative models, resemble System 1. A reasoning feedback loop matches my intuition about how my own brain works (when I’m actually thinking and not in some other brain state like enjoying the moment).
Question for everyone: Do you feel like there is one “missing thing” between these generative models and AGI, and what is it? Or do you think these generative models are not on the path to AGI, however impressive they are?
I’ve been thinking a lot about the “missing thing”. In fact I have some experiments planned, if I ever have the time and compute, to get at least an intuition about system 2 thinking in transformers.
But if you look at the PaLM-paper (and more generally at gwern’s collection of internal monologue examples) it sure looks like deliberate reasoning emerges in very large models.
If there is a “missing thing” I think it is more likely to be something about the representations learned by humans being right off the bat more “gears-level”. Maybe like Hawkin’s reference frames. Some decomposability that enables much more powerful abstraction and that has to be pounded into a NN with millions of examples.
That kind of “missing thing” would impact extrapolation, one-shot-learning, robust system 2 thinking, abstraction, long term planning, causal reasoning, thinking long on hard problems etc.
That makes a lot of sense, and if it’s that’s true then that’s hopeful for interpretability efforts. It would be easier to read inside an ML model if it’s composed of parts that map to real concepts.
It’s pretty difficult to tell intuitively because the human mind is programmed to anthropomorphize. It’s a binary recognition; either it looks 100% human or it doesn’t.
We’re not built to compare 2 different systems that can do some human subroutines but not others. So AI could make a big leap approaching general intelligence, and the lion’s share of that leap could be visible or invisible based on how much the resulting behavior reminds us of ourselves.
Due to the anthropic principle, general intelligence could have a one-in-an-octillion chance of ever randomly evolving, anywhere, ever, and we would still be here observing all the successful steps having happened, because if all the steps didn’t happen then we wouldn’t be here observing anything. There would still be tons of animals like ants and chimpanzees because evolution always creates a ton of alternative “failed” offshoots. So it’s always possible that there’s some logical process that’s necessary for general intelligence, and we’re astronomically unlikely to discover it randomly, through brute forcing or even innovation, until we pinpoint all the exact lines of code in the human brain that distinguishes our intelligence from chimpanzees. But that’s only a possibility, far from a guarantee.
Yoshua Bengio did talk about System 2 Deep Learning at NeurIPS 2019