To me, the recent hardware shortage is very strong evidence that we will not be surprised by a sharp jump in capabilities after a pause, as a result of the pause creating an overhang that eliminates all or nearly all bottlenecks to reaching ASI.
I don’t follow the reasoning here. Shouldn’t a hardware shortage be evidence we will see a spike after a pause?
For example, suppose we pause now for 3 years and during that time NVIDIA releases the RTX5090,6090,7090 which are produced using TSMC’s 3nm, 2nm and 10a processes. Then the amount of compute available at the end of the three year pause will be dramatically higher than it is today. (for reference, the 4090 is 4x better at inference than the 3090). Roughly speaking, then, after your 3 year pause a billion dollar investment will buy 64x as much compute (this is more than the difference between GPT-4 and GPT-3).
Also, a “pause” would most likely only be a cap on the largest training runs. It is unlikely that we’re going to pause all research on current LLM capabilities. Consider that a large part of the “algorithmic progress” in LLM inference speed is driven not by SOTA models, but by hobbyists trying to get LLMs to run faster on their own devices.
This means that in addition to the 64x hardware improvement, we would also get algorithmic improvement (which has historically faster than hardware improvement).
That means at the end of a 3 year pause, an equal cost run would be not 64x but 4096x larger.
Finally, LLMs have already reached the point where they can be reasonably expected to speed up economic growth. Given their economic value will become more obvious over time,the longer we pause, the more we can expect that the largest actors will be willing to spend on a single run. It’s hard to put an estimate on this, but consider that historically the largest runs have been increasing at 3x/year. Even if we conservatively estimate 2x per year, that gives us an additional 8x at the end of our 3 year pause. This now gives us a factor of 32k at the end of our 3 year pause.
Even if you don’t buy that “Most alignment progress will happen from studying closer-to-superhuman models”, surely you believe that “large discontinuous changes are risky” and a factor of 32,000x is a “large discontinuous change”.
We ran into a hardware shortage during a period of time where there was no pause, which is evidence that the hardware manufacturer was behaving conservatively. If they’re behaving conservatively during a boom period like this, it’s not crazy to think they might be even more conservative in terms of novel R&D investment & ramping up manufacturing capacity if they suddenly saw dramatically reduced demand from their largest customers.
For example, suppose we pause now for 3 years and during that time NVIDIA releases the RTX5090,6090,7090 which are produced using TSMC’s 3nm, 2nm and 10a processes.
This and the rest of your comment seems to have ignored the rest of my post (see: multiple inputs to progress, all of which seem sensitive to “demand” from e.g. AGI labs), so I’m not sure how to respond. Do you think NVIDIA’s planning is totally decoupled from anticipated demand for their products? That seems kind of crazy, but that’s the scenario you seem to be describing. Big labs are just going to continue to increase their willingness-to-spend along a smooth exponential for as a long as the pause lasts? What if the pause lasts 10 years?
If you think my model of how inputs to capabilities progress are sensitive to demand for those inputs from AGI labs is wrong, then please argue so directly, or explain how your proposed scenario is compatible with it.
We ran into a hardware shortage during a period of time where there was no pause, which is evidence that the hardware manufacturer was behaving conservatively.
Alternative hypothesis, there are physical limits on how fast you can build things.
Also, NVIDIA currently has a monopoly on “decent AI accelerator you can actually buy”. Part of the “shortage” is just the standard economic result that a monopoly produces less of something to increase profits.
This monopoly will not last forever, so in that sense we are currently in hardware “underhang”.
This and the rest of your comment seems to have ignored the rest of my post (see: multiple inputs to progress, all of which seem sensitive to “demand”
Nvidia doesn’t just make AGI accelerators. They are are video game graphics card company.
And even if we pause large training runs, demand for inference of existing models will continue to increase.
If you think my model of how inputs to capabilities progress are sensitive to demand for those inputs from AGI labs is wrong, then please argue so directly, or explain how your proposed scenario is compatible with it.
This is me arguing directly.
The model “all demand for hardware is driven by a handful of labs training cutting edge models” is completely implausible. It doesn’t explain how we got the hardware in the first place (video games) and it ignores the fact that there exist uses for AI acceleration hardware other than training cutting-edge models.
I don’t follow the reasoning here. Shouldn’t a hardware shortage be evidence we will see a spike after a pause?
For example, suppose we pause now for 3 years and during that time NVIDIA releases the RTX5090,6090,7090 which are produced using TSMC’s 3nm, 2nm and 10a processes. Then the amount of compute available at the end of the three year pause will be dramatically higher than it is today. (for reference, the 4090 is 4x better at inference than the 3090). Roughly speaking, then, after your 3 year pause a billion dollar investment will buy 64x as much compute (this is more than the difference between GPT-4 and GPT-3).
Also, a “pause” would most likely only be a cap on the largest training runs. It is unlikely that we’re going to pause all research on current LLM capabilities. Consider that a large part of the “algorithmic progress” in LLM inference speed is driven not by SOTA models, but by hobbyists trying to get LLMs to run faster on their own devices.
This means that in addition to the 64x hardware improvement, we would also get algorithmic improvement (which has historically faster than hardware improvement).
That means at the end of a 3 year pause, an equal cost run would be not 64x but 4096x larger.
Finally, LLMs have already reached the point where they can be reasonably expected to speed up economic growth. Given their economic value will become more obvious over time,the longer we pause, the more we can expect that the largest actors will be willing to spend on a single run. It’s hard to put an estimate on this, but consider that historically the largest runs have been increasing at 3x/year. Even if we conservatively estimate 2x per year, that gives us an additional 8x at the end of our 3 year pause. This now gives us a factor of 32k at the end of our 3 year pause.
Even if you don’t buy that “Most alignment progress will happen from studying closer-to-superhuman models”, surely you believe that “large discontinuous changes are risky” and a factor of 32,000x is a “large discontinuous change”.
We ran into a hardware shortage during a period of time where there was no pause, which is evidence that the hardware manufacturer was behaving conservatively. If they’re behaving conservatively during a boom period like this, it’s not crazy to think they might be even more conservative in terms of novel R&D investment & ramping up manufacturing capacity if they suddenly saw dramatically reduced demand from their largest customers.
This and the rest of your comment seems to have ignored the rest of my post (see: multiple inputs to progress, all of which seem sensitive to “demand” from e.g. AGI labs), so I’m not sure how to respond. Do you think NVIDIA’s planning is totally decoupled from anticipated demand for their products? That seems kind of crazy, but that’s the scenario you seem to be describing. Big labs are just going to continue to increase their willingness-to-spend along a smooth exponential for as a long as the pause lasts? What if the pause lasts 10 years?
If you think my model of how inputs to capabilities progress are sensitive to demand for those inputs from AGI labs is wrong, then please argue so directly, or explain how your proposed scenario is compatible with it.
Alternative hypothesis, there are physical limits on how fast you can build things.
Also, NVIDIA currently has a monopoly on “decent AI accelerator you can actually buy”. Part of the “shortage” is just the standard economic result that a monopoly produces less of something to increase profits.
This monopoly will not last forever, so in that sense we are currently in hardware “underhang”.
Nvidia doesn’t just make AGI accelerators. They are are video game graphics card company.
And even if we pause large training runs, demand for inference of existing models will continue to increase.
This is me arguing directly.
The model “all demand for hardware is driven by a handful of labs training cutting edge models” is completely implausible. It doesn’t explain how we got the hardware in the first place (video games) and it ignores the fact that there exist uses for AI acceleration hardware other than training cutting-edge models.