I think this is exactly right. The phrasing is a little confusing. I’d say “LLMs can’t solve truly novel problems”.
But the implication that this is a slow route or dead-end for AGI is wrong. I think it’s going to be pretty easy to scaffold LLMs into solving novel problems. I could be wrong, but don’t bet heavily on it unless you happen to know way more about cognitive psychology and LLMs in combination than I do. it would be foolish to make a plan for survival that relies on this being a major delay.
I can’t convince you of this without describing exactly how I think this will be relatively straightforward, and I’m not ready to advance capabilities in this direction yet. I think language model agents are probably our best shot at alignment, so we should probably actively work on advancing them to AGI; but I’m not sure enough yet to start publishing my best theories on how to do that.
Back to the possibly confusing phrasing Chollet uses: I think he’s using Piaget’s definition of intelligence as “what you do when you don’t know what to do” (he quotes this in the interview). That’s restricting it to solving problems you haven’t memorized an approach to. That’s not how most people use the word intelligence.
When he says LLMs “just memorize”, he’s including memorizing programs or approaches to problems, and they can plug the variables of this particular variant of the problem in to those memorized programs/approaches. I think the question “well maybe that’s all you need to do” raised by Patel is appropriate; it’s clear they can’t do enough of this yet, but it’s unclear if further progress will get them to another level of abstraction of an approach so abstract and general that it can solve almost any problem.
I think he’s on the wrong track with the “discrete program search” because I see more human-like solutions that may be lower-hanging fruit, but I wouldn’t bet his approach won’t work. I’m starting to think that there are many approaches to general intelligence, and a lot of them just aren’t that hard. We’d like to think our intelligence is unique, magical, and special, but it’s looking like it’s not. Or at least, we should assume it’s not if we want to plan for other intelligences well enough to survive. So I think alignment workers should assume LLMs with or without scaffolding might very well pass this hurdle fairly quickly.
I think this is exactly right. The phrasing is a little confusing. I’d say “LLMs can’t solve truly novel problems”.
But the implication that this is a slow route or dead-end for AGI is wrong. I think it’s going to be pretty easy to scaffold LLMs into solving novel problems. I could be wrong, but don’t bet heavily on it unless you happen to know way more about cognitive psychology and LLMs in combination than I do. it would be foolish to make a plan for survival that relies on this being a major delay.
I can’t convince you of this without describing exactly how I think this will be relatively straightforward, and I’m not ready to advance capabilities in this direction yet. I think language model agents are probably our best shot at alignment, so we should probably actively work on advancing them to AGI; but I’m not sure enough yet to start publishing my best theories on how to do that.
Back to the possibly confusing phrasing Chollet uses: I think he’s using Piaget’s definition of intelligence as “what you do when you don’t know what to do” (he quotes this in the interview). That’s restricting it to solving problems you haven’t memorized an approach to. That’s not how most people use the word intelligence.
When he says LLMs “just memorize”, he’s including memorizing programs or approaches to problems, and they can plug the variables of this particular variant of the problem in to those memorized programs/approaches. I think the question “well maybe that’s all you need to do” raised by Patel is appropriate; it’s clear they can’t do enough of this yet, but it’s unclear if further progress will get them to another level of abstraction of an approach so abstract and general that it can solve almost any problem.
I think he’s on the wrong track with the “discrete program search” because I see more human-like solutions that may be lower-hanging fruit, but I wouldn’t bet his approach won’t work. I’m starting to think that there are many approaches to general intelligence, and a lot of them just aren’t that hard. We’d like to think our intelligence is unique, magical, and special, but it’s looking like it’s not. Or at least, we should assume it’s not if we want to plan for other intelligences well enough to survive. So I think alignment workers should assume LLMs with or without scaffolding might very well pass this hurdle fairly quickly.