What could plausibly take us from now to AGI within 10 years?
A friend shared the following question on Facebook:
So, I’ve seen multiple articles recently by people who seem well-informed that claim that AGI (artificial general intelligence, aka software that can actually think and is creative) in less than 10 years, and I find that baffling, and am wondering if there’s anything I’m missing. Sure, modern AI like ChatGPT are impressive—they can do utterly amazing search engine-like things, but they aren’t creative at all.
The clearest example of this I’ve seen comes from people’s experiences with AI writing code. From what I’ve read, AI can do exceptionally well with this task, but only if there are examples of the needed sort of code online that it can access or was trained on, and if it lacks this, it’s accuracy is quite bad with easy problems and essentially non-existent with problems that are at all difficult. This clearly says to me that current AI are glorified very impressive search engines, and that’s nowhere near what I’d consider AGI and doesn’t look like it could become AGI.
Am I missing something?
I replied with some of my thoughts as follows:
I have also been a little confused by the shortness of some of the AGI timelines that people have been proposing, and I agree that there are types of creativity that they’re missing, but saying that they’re not creative at all sounds too strong. I’ve been using Claude as a co-writer partner for some fiction and it has felt creative to me. Also e.g. the example of this conversation that someone had with it.
In 2017 I did a small literature review on human expertise, which to me suggested that expertise can broadly be divided into two interacting components: pattern recognition and mental simulation. Pattern recognition is what current LLMs do, essentially. Mental simulation is the bit that they’re missing—if a human programmer is facing a novel programming challenge, they can attack it from first principles and simulate the program execution in their head to see what needs to be done.
The big question would then be something like “how hard would it be to add mental simulation to LLMs”. Some indications that it wouldn’t necessarily be that hard:
* In humans, while they are distinct capabilities, the two also seem to be intertwined. If I’m writing a social media comment and I try to mentally simulate how it will be received, I can do it because I have a rich library of patterns about how different kinds of comments will be received by different readers. If write something that triggers a pattern-detector that goes “uh-oh, that wouldn’t be received well”, I can rewrite it until it passes my mental simulation. That suggests that there would be a natural connection between the two. * There are indications that current LLMs may already be doing something like internal simulation though not being that great at it. Like in the “mouse mastermind” vignette, it certainly intuitively feels like Claude has some kind of consistent internal model of what’s going on. People have also e.g. trained LLMs to play games like Othello and found that the resulting network has an internal representation of the game board ( https://www.lesswrong.com/posts/nmxzr2zsjNtjaHh7x/actually-othello-gpt-has-a-linear-emergent-world ). * There have also been various attempts at explicitly combining an LLM-based component with a component that does something like simulation. E.g. DeepMind trained a hybrid LLM-theorem prover system that reached silver medal-level performance on this year’s International Mathematics Olympiad ( https://deepmind.google/discover/blog/ai-solves-imo-problems-at-silver-medal-level/ ), where the theorem prover component maintains a type of state over the math problem as it’s being worked on. * Iterative improvements like chain-of-thought reasoning are also taking LLMs in the direction of being able to apply more novel reasoning in domains such as math. Mathematician Terry Tao commented the following about giving the recent GPT-o1 model research-level math tasks to work on:
> The experience seemed roughly on par with trying to advise a mediocre, but not completely incompetent, (static simulation of a) graduate student. However, this was an improvement over previous models, whose capability was closer to an actually incompetent (static simulation of a) graduate student. It may only take one or two further iterations of improved capability (and integration with other tools, such as computer algebra packages and proof assistants) until the level of “(static simulation of a) competent graduate student” is reached, at which point I could see this tool being of significant use in research level tasks.
* There have also been other papers trying out various techniques such as “whiteboard of thought” ( https://whiteboard.cs.columbia.edu/ ) where an LLM, when being presented with visual problems in verbal format, explicitly generates visual representations of the verbal description to use as an aid in its reasoning. It feels like a relatively obvious idea would be to roll out these kinds of approaches into future LLM architectures, teaching them to generate “mental images” of whatever task they were told to work on. This could then be used as part of an internal simulation. * There’s an evolutionary argument that the steps from “pure pattern recognition” to “pattern recognition with mental simulation added” might be relatively simple and not require that much in the fundamental breakthroughs, since evolution managed to find it in humans and in humans those abilities seem to be relatively continuous with each other. So we might expect all of these iterative improvements to take us pretty smoothly toward AGI.
Here’s my brief pitch, starting with your point about simulation:
The strength and flexibility of LLMs probably opens up several more routes toward cognitive completeness and what we’d consider impressive creativity.
LLMs can use chain-of-thought sequential processing to do a type of mental simulation. If they are prompted to, or if they “prompt themselves” in a chain of thought system, they can access a rich world model to simulate how different actions are likely to play out. They have to put everything in language, although visual and other modalities can be added either through things like the whiteboard of thought, or by using CoT training directly on those modalities in multimodal foundation models. But language already summarizes a good deal of world models across many modalities, so those improvements may not be necessary.
The primary change that will make LLMs more “creative” in your friends’ sense is letting them think longer and using strategy and training to organize that thinking. There are two cognitive capacities needed to do this. There is no barrier to progress in either direction; they just haven’t received much attention yet.
LLMs don’t have any episodic memory, “snapshot” memory for important experiences. And They’re severely lacking executive functioning, the capacity to keep ourselves on-track and strategically direct our cognition. A human with those impairments would be very little use for complex tasks, let alone doing novel work we’d consider deeply creative.
Both of those things seem actually pretty easy to add. Vector-based databases aren’t quite good enough to be very useful, but they will be improved. One route is a straightforward, computationally-efficient improvement based on human brain function that I won’t mention even though work is probably underway on it somewhere. And there are probably other equally good routes.
The chain-of-thought training applied to o1, r1, Marco o1, and QwQ (and probably soon a whole bunch more) improves organization of chains of thought, adding some amount of executive function. Scaffolding in prompts for things like “where are you in the task? Is this making progress toward the goal? Should we try a different approach?” etc is also possible. This will work better when combined with episodic memory; a human without it couldn’t organize their progress through a complex task—but LLMs now have large context windows that are like better-than-human working memory systems, so better episodic memory might not even be necessary for dramatic improvements.
That’s just one route to “Real AGI” from LLMs/foundation models. There are probably others that are just as easy. Foundation models can now do almost everything humans can in the short term. Making their cognition cumulative like ours seems like more of an unblocking and using their capacities more strategically and effectively, rather than adding any real new cognitive abilities.
Continuous learning, through better episodic memory and/or fine-tuning for facts/skills judged as useful is another low-hanging fruit.
Hoping that we’re more than a decade from transformative AGI now seems wildly optimistic to me. There could be dramatic roadblocks I haven’t foreseen, but most of those would just push it past three years. It could take more than a decade, but banking on that leaves us unprepared for the very short timelines that now seem fairly likely.
While the short timelines are scary, there are also large advantages to this route to AGI, including a relatively slow takeoff and the way that LLMs are almost an oracle AI trained largely to follow instructions. But that’s another story.
That’s a bit more than I meant to write; I’ve been trying to refine an intuitive explanation of why we may be spitting distance from real, transformative AGI, and that served as a useful prompt.
Hoping that we’re more than a decade from transformative AGI now seems wildly optimistic to me. There could be dramatic roadblocks I haven’t foreseen, but most of those would just push it past three years.
Self-driving cars seem like a useful reference point. Back when cars got unexpectedly good performance at the 2005 and 2007 DARPA grand challenges, there was a lot of hype about how self-driving cars were just around the corner now that they had demonstrated having the basic capability. 17 years later, we’re only at this point (Wikipedia):
As of late 2024, no system has achieved full autonomy (SAE Level 5). In December 2020, Waymo was the first to offer rides in self-driving taxis to the public in limited geographic areas (SAE Level 4),[7] and as of April 2024 offers services in Arizona (Phoenix) and California (San Francisco and Los Angeles). [...] In July 2021, DeepRoute.ai started offering self-driving taxi rides in Shenzhen, China. Starting in February 2022, Cruise offered self-driving taxi service in San Francisco,[11] but suspended service in 2023. In 2021, Honda was the first manufacturer to sell an SAE Level 3 car,[12][13][14] followed by Mercedes-Benz in 2023.
I admit, I’d probably call self-driving cars at this point a solved or nearly-solved problem by Waymo, and the big reason why self-driving cars only now are taking off is basically because of regulatory and liability issues, and I consider a lot of the self-driving car slowdown as evidence that regulation can work to slow down a technology substantially.
It seems to me that o1 and deepseek already do a bunch of the “mental simulation” kind of reasoning, and even previous LLMs did so a good amount if you prompted them to think in chain-of-thoughts, so the core point fell a bit flat for me.
Thanks, that’s helpful. My impression from o1 is that it does something that could be called mental simulation for domains like math where the “simulation” can in fact be represented with just writing (or equations more specifically). But I think that writing is only an efficient format for mental simulation for a very small number of domains.
What could plausibly take us from now to AGI within 10 years?
A friend shared the following question on Facebook:
I replied with some of my thoughts as follows:
Here’s my brief pitch, starting with your point about simulation:
The strength and flexibility of LLMs probably opens up several more routes toward cognitive completeness and what we’d consider impressive creativity.
LLMs can use chain-of-thought sequential processing to do a type of mental simulation. If they are prompted to, or if they “prompt themselves” in a chain of thought system, they can access a rich world model to simulate how different actions are likely to play out. They have to put everything in language, although visual and other modalities can be added either through things like the whiteboard of thought, or by using CoT training directly on those modalities in multimodal foundation models. But language already summarizes a good deal of world models across many modalities, so those improvements may not be necessary.
The primary change that will make LLMs more “creative” in your friends’ sense is letting them think longer and using strategy and training to organize that thinking. There are two cognitive capacities needed to do this. There is no barrier to progress in either direction; they just haven’t received much attention yet.
LLMs don’t have any episodic memory, “snapshot” memory for important experiences. And They’re severely lacking executive functioning, the capacity to keep ourselves on-track and strategically direct our cognition. A human with those impairments would be very little use for complex tasks, let alone doing novel work we’d consider deeply creative.
Both of those things seem actually pretty easy to add. Vector-based databases aren’t quite good enough to be very useful, but they will be improved. One route is a straightforward, computationally-efficient improvement based on human brain function that I won’t mention even though work is probably underway on it somewhere. And there are probably other equally good routes.
The chain-of-thought training applied to o1, r1, Marco o1, and QwQ (and probably soon a whole bunch more) improves organization of chains of thought, adding some amount of executive function. Scaffolding in prompts for things like “where are you in the task? Is this making progress toward the goal? Should we try a different approach?” etc is also possible. This will work better when combined with episodic memory; a human without it couldn’t organize their progress through a complex task—but LLMs now have large context windows that are like better-than-human working memory systems, so better episodic memory might not even be necessary for dramatic improvements.
This is spelled out a little more in Capabilities and alignment of LLM cognitive architectures, although that isn’t as clear or compelling as I’d like. It looks to me like progress is happening apace on that direction.
That’s just one route to “Real AGI” from LLMs/foundation models. There are probably others that are just as easy. Foundation models can now do almost everything humans can in the short term. Making their cognition cumulative like ours seems like more of an unblocking and using their capacities more strategically and effectively, rather than adding any real new cognitive abilities.
Continuous learning, through better episodic memory and/or fine-tuning for facts/skills judged as useful is another low-hanging fruit.
Hoping that we’re more than a decade from transformative AGI now seems wildly optimistic to me. There could be dramatic roadblocks I haven’t foreseen, but most of those would just push it past three years. It could take more than a decade, but banking on that leaves us unprepared for the very short timelines that now seem fairly likely.
While the short timelines are scary, there are also large advantages to this route to AGI, including a relatively slow takeoff and the way that LLMs are almost an oracle AI trained largely to follow instructions. But that’s another story.
That’s a bit more than I meant to write; I’ve been trying to refine an intuitive explanation of why we may be spitting distance from real, transformative AGI, and that served as a useful prompt.
Self-driving cars seem like a useful reference point. Back when cars got unexpectedly good performance at the 2005 and 2007 DARPA grand challenges, there was a lot of hype about how self-driving cars were just around the corner now that they had demonstrated having the basic capability. 17 years later, we’re only at this point (Wikipedia):
And self-driving capability should be vastly easier than general intelligence. Like self-driving, transformative AI also requires reliable worst-case performance rather than just good average-case performance, and there’s usually a surprising amount of detail involved that you need to sort out before you get to that point.
I admit, I’d probably call self-driving cars at this point a solved or nearly-solved problem by Waymo, and the big reason why self-driving cars only now are taking off is basically because of regulatory and liability issues, and I consider a lot of the self-driving car slowdown as evidence that regulation can work to slow down a technology substantially.
(Hmm I was expecting that this would get more upvotes. Too obvious? Not obvious enough?)
It seems to me that o1 and deepseek already do a bunch of the “mental simulation” kind of reasoning, and even previous LLMs did so a good amount if you prompted them to think in chain-of-thoughts, so the core point fell a bit flat for me.
Thanks, that’s helpful. My impression from o1 is that it does something that could be called mental simulation for domains like math where the “simulation” can in fact be represented with just writing (or equations more specifically). But I think that writing is only an efficient format for mental simulation for a very small number of domains.