Incidentally, I think GPT-3 is great evidence that human-legible learning algorithms are up to the task of directly learning and using a common-sense world-model. I’m not saying that GPT-3 is necessarily directly on the path to AGI; instead I’m saying, How can you look at GPT-3 (a simple learning algorithm with a ridiculously simple objective) and then say, “Nope! AGI is way beyond what human-legible learning algorithms can do! We need a totally different path!”?
I think the response would be, “GPT-3 may have learned an awesome general common-sense world-model, but it took 300,000,000 tokens of training to do so. AI won’t be transformative until it can learn quickly/data-efficiently. (Or until we have enough compute to train it slowly/inefficiently on medium or long-horizon tasks, which is far in the future.)”
A kinda generic answer is: (1) Transformers were an advance over previous learning algorithms, and by the same token I expect that yet-to-be-invented learning algorithms will be an advance over Transformers; (2) Sample-efficient learning is AFAICT a hot area that lots of people are working on; (3) We do in fact actually have impressively sample-efficient algorithms even if they’re not as well-developed and scalable as others at the moment—see my discussion of analysis-by-synthesis; (4) Given that predictive learning offers tons of data, it’s not obvious how important sample-efficiency is.
More detailed answer: I agree that in the “intelligence via online learning” paradigm I mentioned, you really want to see something once and immediately commit it to memory. Hard to carry on a conversation otherwise! The human brain has two main tricks for this (that I know of).
There’s a giant structured memory (predictive world-model) in the neocortex, and a much smaller unstructured memory in the hippocampus, and the latter is basically just an auto-associative memory (with a pattern separator to avoid cross-talk) that memorizes things. Then it can replay it when appropriate. And just like replay learning in ML, or like doing multiple passes through your training data in ML, relevant information can gradually transfer from the unstructured memory to the structured one by repeated replays.
Because the structured memory is in the analysis-by-synthesis paradigm (i.e. searching for a generative model that matches the data), it inherently needs less training data, because its inductive biases are a closer match to reality. It’s a harder search problem to build the right generative model when you’re learning, and it’s a harder search problem to find the right generative model at inference time, but once you get it, it generalizes better and takes you farther. For example, you can “train on no data whatsoever”—just stare into space for a while, thinking about the problem, and wind up learning something new. This is only possible because you have a space of generative models, so you can run internal experiments. How’s that for sample efficiency?!
AlphaStar and GPT-3 don’t do analysis-by-synthesis—well, they weren’t designed to do it, although my hunch is that GPT-3 is successful by doing it to a limited extent (this may be related to the Hopfield network thing). But we do have algorithms at an earlier stage of development / refinement / scaling-up that are based on those principles, and they are indeed very highly sample-efficient, and in the coming years I expect that they’ll be used more widely in AI.
To make sure I understand: you are saying (a) that our AIs are fairly likely to get significantly more sample-efficient in the near future, and (b) even if they don’t, there’s plenty of data around.
I think (b) isn’t a good response if you think that transformative AI will probably need to be human brain sized and you believe the scaling laws and you think that short-horizon training won’t be enough. (Because then we’ll need something like 10^30+ FLOP to train TAI, which is plausibly reachable in 20 years but probably not in 10. That said, I think short-horizon training might be enough.
I think (a) is a good response, but it faces the objection: Why now? Why should we expect sample-efficiency to get dramatically better in the near future, when it has gotten only very slowly better in the past? (Has it? I’m guessing so, maybe I’m wrong?)
I think the response would be, “GPT-3 may have learned an awesome general common-sense world-model, but it took 300,000,000 tokens of training to do so. AI won’t be transformative until it can learn quickly/data-efficiently. (Or until we have enough compute to train it slowly/inefficiently on medium or long-horizon tasks, which is far in the future.)”
What would you say to that?
Good question!
A kinda generic answer is: (1) Transformers were an advance over previous learning algorithms, and by the same token I expect that yet-to-be-invented learning algorithms will be an advance over Transformers; (2) Sample-efficient learning is AFAICT a hot area that lots of people are working on; (3) We do in fact actually have impressively sample-efficient algorithms even if they’re not as well-developed and scalable as others at the moment—see my discussion of analysis-by-synthesis; (4) Given that predictive learning offers tons of data, it’s not obvious how important sample-efficiency is.
More detailed answer: I agree that in the “intelligence via online learning” paradigm I mentioned, you really want to see something once and immediately commit it to memory. Hard to carry on a conversation otherwise! The human brain has two main tricks for this (that I know of).
There’s a giant structured memory (predictive world-model) in the neocortex, and a much smaller unstructured memory in the hippocampus, and the latter is basically just an auto-associative memory (with a pattern separator to avoid cross-talk) that memorizes things. Then it can replay it when appropriate. And just like replay learning in ML, or like doing multiple passes through your training data in ML, relevant information can gradually transfer from the unstructured memory to the structured one by repeated replays.
Because the structured memory is in the analysis-by-synthesis paradigm (i.e. searching for a generative model that matches the data), it inherently needs less training data, because its inductive biases are a closer match to reality. It’s a harder search problem to build the right generative model when you’re learning, and it’s a harder search problem to find the right generative model at inference time, but once you get it, it generalizes better and takes you farther. For example, you can “train on no data whatsoever”—just stare into space for a while, thinking about the problem, and wind up learning something new. This is only possible because you have a space of generative models, so you can run internal experiments. How’s that for sample efficiency?!
AlphaStar and GPT-3 don’t do analysis-by-synthesis—well, they weren’t designed to do it, although my hunch is that GPT-3 is successful by doing it to a limited extent (this may be related to the Hopfield network thing). But we do have algorithms at an earlier stage of development / refinement / scaling-up that are based on those principles, and they are indeed very highly sample-efficient, and in the coming years I expect that they’ll be used more widely in AI.
To make sure I understand: you are saying (a) that our AIs are fairly likely to get significantly more sample-efficient in the near future, and (b) even if they don’t, there’s plenty of data around.
I think (b) isn’t a good response if you think that transformative AI will probably need to be human brain sized and you believe the scaling laws and you think that short-horizon training won’t be enough. (Because then we’ll need something like 10^30+ FLOP to train TAI, which is plausibly reachable in 20 years but probably not in 10. That said, I think short-horizon training might be enough.
I think (a) is a good response, but it faces the objection: Why now? Why should we expect sample-efficiency to get dramatically better in the near future, when it has gotten only very slowly better in the past? (Has it? I’m guessing so, maybe I’m wrong?)