It depends on how much time there is between the first impactful demonstration of long-horizon task capabilities (doing many jobs) and commoditization of research capable TAI, even with governments waking up during this interval and working to extend it. It might be that by default this is already at least few years, and if the bulk of compute is seized, it extends to even longer. This seems to require long-horizon task capabilities to be found at the limits of scaling, and TAI significantly further.
But we don’t know until it’s tried if even a $3 billion training run won’t already enable long-horizon task capabilities (with appropriate post-training, even if it arrives a bit later), and we don’t know if the first long-horizon task capable AI won’t immediately be capable of research, with no need for further scaling (even if it helps). And if it’s not immediately obvious how to elicit these capabilities with post-training, there will be an overhang of sufficient compute and sufficiently strong base models in many places before the alarm is sounded. If enough of such things align, there won’t be time for anyone to prevent prompt commoditization of research capable TAI. And then there’s ASI 1-2 years later, with the least possible time for anyone to steer any of this.
It depends on how much time there is between the first impactful demonstration of long-horizon task capabilities (doing many jobs) and commoditization of research capable TAI, even with governments waking up during this interval and working to extend it. It might be that by default this is already at least few years, and if the bulk of compute is seized, it extends to even longer. This seems to require long-horizon task capabilities to be found at the limits of scaling, and TAI significantly further.
But we don’t know until it’s tried if even a $3 billion training run won’t already enable long-horizon task capabilities (with appropriate post-training, even if it arrives a bit later), and we don’t know if the first long-horizon task capable AI won’t immediately be capable of research, with no need for further scaling (even if it helps). And if it’s not immediately obvious how to elicit these capabilities with post-training, there will be an overhang of sufficient compute and sufficiently strong base models in many places before the alarm is sounded. If enough of such things align, there won’t be time for anyone to prevent prompt commoditization of research capable TAI. And then there’s ASI 1-2 years later, with the least possible time for anyone to steer any of this.