Here’s my brief pitch, starting with your point about simulation:
The strength and flexibility of LLMs probably opens up several more routes toward cognitive completeness and what we’d consider impressive creativity.
LLMs can use chain-of-thought sequential processing to do a type of mental simulation. If they are prompted to, or if they “prompt themselves” in a chain of thought system, they can access a rich world model to simulate how different actions are likely to play out. They have to put everything in language, although visual and other modalities can be added either through things like the whiteboard of thought, or by using CoT training directly on those modalities in multimodal foundation models. But language already summarizes a good deal of world models across many modalities, so those improvements may not be necessary.
The primary change that will make LLMs more “creative” in your friends’ sense is letting them think longer and using strategy and training to organize that thinking. There are two cognitive capacities needed to do this. There is no barrier to progress in either direction; they just haven’t received much attention yet.
LLMs don’t have any episodic memory, “snapshot” memory for important experiences. And They’re severely lacking executive functioning, the capacity to keep ourselves on-track and strategically direct our cognition. A human with those impairments would be very little use for complex tasks, let alone doing novel work we’d consider deeply creative.
Both of those things seem actually pretty easy to add. Vector-based databases aren’t quite good enough to be very useful, but they will be improved. One route is a straightforward, computationally-efficient improvement based on human brain function that I won’t mention even though work is probably underway on it somewhere. And there are probably other equally good routes.
The chain-of-thought training applied to o1, r1, Marco o1, and QwQ (and probably soon a whole bunch more) improves organization of chains of thought, adding some amount of executive function. Scaffolding in prompts for things like “where are you in the task? Is this making progress toward the goal? Should we try a different approach?” etc is also possible. This will work better when combined with episodic memory; a human without it couldn’t organize their progress through a complex task—but LLMs now have large context windows that are like better-than-human working memory systems, so better episodic memory might not even be necessary for dramatic improvements.
That’s just one route to “Real AGI” from LLMs/foundation models. There are probably others that are just as easy. Foundation models can now do almost everything humans can in the short term. Making their cognition cumulative like ours seems like more of an unblocking and using their capacities more strategically and effectively, rather than adding any real new cognitive abilities.
Continuous learning, through better episodic memory and/or fine-tuning for facts/skills judged as useful is another low-hanging fruit.
Hoping that we’re more than a decade from transformative AGI now seems wildly optimistic to me. There could be dramatic roadblocks I haven’t foreseen, but most of those would just push it past three years. It could take more than a decade, but banking on that leaves us unprepared for the very short timelines that now seem fairly likely.
While the short timelines are scary, there are also large advantages to this route to AGI, including a relatively slow takeoff and the way that LLMs are almost an oracle AI trained largely to follow instructions. But that’s another story.
That’s a bit more than I meant to write; I’ve been trying to refine an intuitive explanation of why we may be spitting distance from real, transformative AGI, and that served as a useful prompt.
Hoping that we’re more than a decade from transformative AGI now seems wildly optimistic to me. There could be dramatic roadblocks I haven’t foreseen, but most of those would just push it past three years.
Self-driving cars seem like a useful reference point. Back when cars got unexpectedly good performance at the 2005 and 2007 DARPA grand challenges, there was a lot of hype about how self-driving cars were just around the corner now that they had demonstrated having the basic capability. 17 years later, we’re only at this point (Wikipedia):
As of late 2024, no system has achieved full autonomy (SAE Level 5). In December 2020, Waymo was the first to offer rides in self-driving taxis to the public in limited geographic areas (SAE Level 4),[7] and as of April 2024 offers services in Arizona (Phoenix) and California (San Francisco and Los Angeles). [...] In July 2021, DeepRoute.ai started offering self-driving taxi rides in Shenzhen, China. Starting in February 2022, Cruise offered self-driving taxi service in San Francisco,[11] but suspended service in 2023. In 2021, Honda was the first manufacturer to sell an SAE Level 3 car,[12][13][14] followed by Mercedes-Benz in 2023.
I admit, I’d probably call self-driving cars at this point a solved or nearly-solved problem by Waymo, and the big reason why self-driving cars only now are taking off is basically because of regulatory and liability issues, and I consider a lot of the self-driving car slowdown as evidence that regulation can work to slow down a technology substantially.
Here’s my brief pitch, starting with your point about simulation:
The strength and flexibility of LLMs probably opens up several more routes toward cognitive completeness and what we’d consider impressive creativity.
LLMs can use chain-of-thought sequential processing to do a type of mental simulation. If they are prompted to, or if they “prompt themselves” in a chain of thought system, they can access a rich world model to simulate how different actions are likely to play out. They have to put everything in language, although visual and other modalities can be added either through things like the whiteboard of thought, or by using CoT training directly on those modalities in multimodal foundation models. But language already summarizes a good deal of world models across many modalities, so those improvements may not be necessary.
The primary change that will make LLMs more “creative” in your friends’ sense is letting them think longer and using strategy and training to organize that thinking. There are two cognitive capacities needed to do this. There is no barrier to progress in either direction; they just haven’t received much attention yet.
LLMs don’t have any episodic memory, “snapshot” memory for important experiences. And They’re severely lacking executive functioning, the capacity to keep ourselves on-track and strategically direct our cognition. A human with those impairments would be very little use for complex tasks, let alone doing novel work we’d consider deeply creative.
Both of those things seem actually pretty easy to add. Vector-based databases aren’t quite good enough to be very useful, but they will be improved. One route is a straightforward, computationally-efficient improvement based on human brain function that I won’t mention even though work is probably underway on it somewhere. And there are probably other equally good routes.
The chain-of-thought training applied to o1, r1, Marco o1, and QwQ (and probably soon a whole bunch more) improves organization of chains of thought, adding some amount of executive function. Scaffolding in prompts for things like “where are you in the task? Is this making progress toward the goal? Should we try a different approach?” etc is also possible. This will work better when combined with episodic memory; a human without it couldn’t organize their progress through a complex task—but LLMs now have large context windows that are like better-than-human working memory systems, so better episodic memory might not even be necessary for dramatic improvements.
This is spelled out a little more in Capabilities and alignment of LLM cognitive architectures, although that isn’t as clear or compelling as I’d like. It looks to me like progress is happening apace on that direction.
That’s just one route to “Real AGI” from LLMs/foundation models. There are probably others that are just as easy. Foundation models can now do almost everything humans can in the short term. Making their cognition cumulative like ours seems like more of an unblocking and using their capacities more strategically and effectively, rather than adding any real new cognitive abilities.
Continuous learning, through better episodic memory and/or fine-tuning for facts/skills judged as useful is another low-hanging fruit.
Hoping that we’re more than a decade from transformative AGI now seems wildly optimistic to me. There could be dramatic roadblocks I haven’t foreseen, but most of those would just push it past three years. It could take more than a decade, but banking on that leaves us unprepared for the very short timelines that now seem fairly likely.
While the short timelines are scary, there are also large advantages to this route to AGI, including a relatively slow takeoff and the way that LLMs are almost an oracle AI trained largely to follow instructions. But that’s another story.
That’s a bit more than I meant to write; I’ve been trying to refine an intuitive explanation of why we may be spitting distance from real, transformative AGI, and that served as a useful prompt.
Self-driving cars seem like a useful reference point. Back when cars got unexpectedly good performance at the 2005 and 2007 DARPA grand challenges, there was a lot of hype about how self-driving cars were just around the corner now that they had demonstrated having the basic capability. 17 years later, we’re only at this point (Wikipedia):
And self-driving capability should be vastly easier than general intelligence. Like self-driving, transformative AI also requires reliable worst-case performance rather than just good average-case performance, and there’s usually a surprising amount of detail involved that you need to sort out before you get to that point.
I admit, I’d probably call self-driving cars at this point a solved or nearly-solved problem by Waymo, and the big reason why self-driving cars only now are taking off is basically because of regulatory and liability issues, and I consider a lot of the self-driving car slowdown as evidence that regulation can work to slow down a technology substantially.