Summary: Superintelligence in January-August, 2026. Paradise or mass death, shortly thereafter.
This is the shortest timeline proposed in these answers so far. My estimate (guess) is that there’s only 20% of this coming true, but it looks feasible as of now. I can’t honestly assert it as fact, but I will say it is possible.
It’s a standard intelligence explosion scenario: with only human effort, the capacities of our AIs double every two years. Once AI gets good enough to do half the work, we double every one year. Once we’ve done that for a year, our now double-smart AIs help us double in six months. Then we double in three months, then six weeks.… to perfect ASI software, running at the the limits of our hardware, in a finite time. Then the ASI does what it wants, and we suffer what we must.
I hear you say “Carl, this argument is as old as the hills. It hasn’t ever come true, why bring it up now?” The answer is, I bring it up because it seems to be happening.
For at least six months now, we’ve had software assistants that can roughly double the productivity of software development. At least at the software company where I work, people using AI (including me) are seeing huge increases in effectiveness. I assume the same is true of AI software development, and that AI labs are using this technology as much as they can.
In the last few months, there’s been a perceptible increase in the speed of releases of better models. I think this is a visible (subjective, debatable) sign of an intelligence explosion starting up.
So I think we’re somewhere in the “doubling in one year” phase of the explosion. If we’re halfway through that year, the singularity is due in August 2026. If we’re near the end of that year, the date is January 2026.
There are lots of things that might go wrong with this scenario, and thereby delay the intelligence explosion. I will mention a few, so you don’t have to.
First, the government might stop the explosion, by banning AI being used for the development of AI. Or perhaps the management of all major AI labs will spontaneously not be so foolish as to. This will delay the problem for an unknown time.
Second, the scenario has an extremely naive model of intelligence explosion microeconomics. It assumes that one doubling of “smartness” produces one doubling of speed. In Yudkowsky’s original scenario, AIs were doing all the work of development, and this might be a sensible assumption. But what has actually happened is that successive generations of AI can handle larger and larger tasks, before they go off the rails. And they can handle these tasks far faster than humans. So the way we work now is that we ask the AI to do some small task, and bang, it’s done. It seems like testing is showing that current AIs can do things that would take a human up to an hour or two. Perhaps the next generation will be able to do tasks up to four hours. The model assumes that this allows a twofold speedup, then fourfold, etc. But this assumption is unsupported.
Third, the scenario assumes that near-term hardware is sufficient for superintelligence. There isn’t time for the accelerating loop to take effect in hardware. Even if design was instant, the physical processes of mask making, lithography, testing, yield optimization and mass production take more than a year. The chips that the ASI will run on in mid-2026 have their design almost done now, at the end of 2024. So we won’t be able to get to ASI, if the ASI requires many orders of magnitude more FLOPs than current models. Instead, we’ll have to wait until the AI designs future generations of semiconductor technology. This will delay matters by years (if using humans to build things) or hours (if using nanotechnology.)
(I don’t think the hardware limit is actually much of a problem; AIs have recently stopped scaling in numbers of parameters and size of training data. Good engineers are constantly figuring out how to pack more intelligence into the same amount of computation. And the human brain provides an existence proof that human-level intelligence requires much less training data. Like Mr. Helm-Burger above, I think human-equivalent cognition is around 10^15 Flops. But reasonable people disagree with me.)
For at least six months now, we’ve had software assistants that can roughly double the productivity of software development.
Is this the consensus view? I’ve seen people saying that those assistants give 10% productivity improvement, at best.
In the last few months, there’s been a perceptible increase in the speed of releases of better models.
On the other hand, the schedules for headline releases (GPT-5, Claude 3.5 Opus) continue to slip, and there are anonymous reports of diminishing returns from scaling. The current moment is interesting in that there are two essentially opposite prevalent narratives barely interacting with each other.
Is this the consensus view? I think it’s generally agreed that software development has been sped up. A factor of two is ambitious! But that’s what it seems to me, and I’ve measured three examples of computer vision programming, each taking an hour or two, by doing them by hand and then with machine assistance. The machines are dumb and produce results that require rewriting. But my code is also inaccurate on a first try. I don’t have any references where people agree with me. And this may not apply to AI programming in general.
You ask about “anonymous reports of diminishing returns to scaling.” I have also heard these reports, direct from a friend who is a researcher inside a major lab. But note that this does not imply a diminished rate of progress, since there are other ways to advance besides making LLMs bigger. O1 and o3 indicate the payoffs to be had by doing things other than pure scaling. If there are forms of progress available to cleverness, then the speed of advance need not require scaling.
The lack of reliability eats away a huge amount of productivity. Everything should be double-checked, and with higher capabilities it becomes even harder, and we need to think more about the subtle ways that their output is wrong. Unknown unknowns are also always a factor, but if o3 type models can be trained in less verifiable problems, and not insanely compute heavy, then 2026 is actually a reasonable guess.
Summary: Superintelligence in January-August, 2026. Paradise or mass death, shortly thereafter.
This is the shortest timeline proposed in these answers so far. My estimate (guess) is that there’s only 20% of this coming true, but it looks feasible as of now. I can’t honestly assert it as fact, but I will say it is possible.
It’s a standard intelligence explosion scenario: with only human effort, the capacities of our AIs double every two years. Once AI gets good enough to do half the work, we double every one year. Once we’ve done that for a year, our now double-smart AIs help us double in six months. Then we double in three months, then six weeks.… to perfect ASI software, running at the the limits of our hardware, in a finite time. Then the ASI does what it wants, and we suffer what we must.
I hear you say “Carl, this argument is as old as the hills. It hasn’t ever come true, why bring it up now?” The answer is, I bring it up because it seems to be happening.
For at least six months now, we’ve had software assistants that can roughly double the productivity of software development. At least at the software company where I work, people using AI (including me) are seeing huge increases in effectiveness. I assume the same is true of AI software development, and that AI labs are using this technology as much as they can.
In the last few months, there’s been a perceptible increase in the speed of releases of better models. I think this is a visible (subjective, debatable) sign of an intelligence explosion starting up.
So I think we’re somewhere in the “doubling in one year” phase of the explosion. If we’re halfway through that year, the singularity is due in August 2026. If we’re near the end of that year, the date is January 2026.
There are lots of things that might go wrong with this scenario, and thereby delay the intelligence explosion. I will mention a few, so you don’t have to.
First, the government might stop the explosion, by banning AI being used for the development of AI. Or perhaps the management of all major AI labs will spontaneously not be so foolish as to. This will delay the problem for an unknown time.
Second, the scenario has an extremely naive model of intelligence explosion microeconomics. It assumes that one doubling of “smartness” produces one doubling of speed. In Yudkowsky’s original scenario, AIs were doing all the work of development, and this might be a sensible assumption. But what has actually happened is that successive generations of AI can handle larger and larger tasks, before they go off the rails. And they can handle these tasks far faster than humans. So the way we work now is that we ask the AI to do some small task, and bang, it’s done. It seems like testing is showing that current AIs can do things that would take a human up to an hour or two. Perhaps the next generation will be able to do tasks up to four hours. The model assumes that this allows a twofold speedup, then fourfold, etc. But this assumption is unsupported.
Third, the scenario assumes that near-term hardware is sufficient for superintelligence. There isn’t time for the accelerating loop to take effect in hardware. Even if design was instant, the physical processes of mask making, lithography, testing, yield optimization and mass production take more than a year. The chips that the ASI will run on in mid-2026 have their design almost done now, at the end of 2024. So we won’t be able to get to ASI, if the ASI requires many orders of magnitude more FLOPs than current models. Instead, we’ll have to wait until the AI designs future generations of semiconductor technology. This will delay matters by years (if using humans to build things) or hours (if using nanotechnology.)
(I don’t think the hardware limit is actually much of a problem; AIs have recently stopped scaling in numbers of parameters and size of training data. Good engineers are constantly figuring out how to pack more intelligence into the same amount of computation. And the human brain provides an existence proof that human-level intelligence requires much less training data. Like Mr. Helm-Burger above, I think human-equivalent cognition is around 10^15 Flops. But reasonable people disagree with me.)
Is this the consensus view? I’ve seen people saying that those assistants give 10% productivity improvement, at best.
On the other hand, the schedules for headline releases (GPT-5, Claude 3.5 Opus) continue to slip, and there are anonymous reports of diminishing returns from scaling. The current moment is interesting in that there are two essentially opposite prevalent narratives barely interacting with each other.
Is this the consensus view? I think it’s generally agreed that software development has been sped up. A factor of two is ambitious! But that’s what it seems to me, and I’ve measured three examples of computer vision programming, each taking an hour or two, by doing them by hand and then with machine assistance. The machines are dumb and produce results that require rewriting. But my code is also inaccurate on a first try. I don’t have any references where people agree with me. And this may not apply to AI programming in general.
You ask about “anonymous reports of diminishing returns to scaling.” I have also heard these reports, direct from a friend who is a researcher inside a major lab. But note that this does not imply a diminished rate of progress, since there are other ways to advance besides making LLMs bigger. O1 and o3 indicate the payoffs to be had by doing things other than pure scaling. If there are forms of progress available to cleverness, then the speed of advance need not require scaling.
The lack of reliability eats away a huge amount of productivity. Everything should be double-checked, and with higher capabilities it becomes even harder, and we need to think more about the subtle ways that their output is wrong. Unknown unknowns are also always a factor, but if o3 type models can be trained in less verifiable problems, and not insanely compute heavy, then 2026 is actually a reasonable guess.