Nice post! You’ve probably seen these already, but for the benefit of other readers, check out this awesome graph from the highly relevant OpenPhil report by Roodman:
More discussion of it here, blue lines added by me:
The red line is real historic GWP data; the splay of grey shades that continues it is the splay of possible futures calculated by the model. The median trajectory is the black line.
I messed around with a ruler to make some rough calculations, marking up the image with blue lines as I went. The big blue line indicates the point on the median trajectory where GWP is 10x what is was in 2019. Eyeballing it, it looks like it happens around 2040, give or take a year. The small vertical blue line indicates the year 2037. The small horizontal blue line indicates GWP in 2037 on the median trajectory.
Thus, it seems that between 2037 and 2040 on the median trajectory, GWP doubles. Thus TAI arrives around 2037 on the median trajectory.
...
Also: You say:
Earth’s best [patterning-thinking-machines] are still significantly slower to learn and to generalize than humans are (in terms of the number of examples they need to see), so for specialized intellectual labour the training involved would currently be prohibitive.
I’d be interested to dig into this claim more. What exactly is the claim, and what is the justification for it? If the claim is something like “For most tasks, the thinking machines seem to need 0 to 3 orders of magnitude more experience on the task before they equal human performance” then I tentatively agree. But if it’s instead 6 to 9 OOMs, or even just a solid 3 OOMs, I’d say “citation needed!”
Also, note that the current models are 3+ OOMs smaller than the human brain, and it’s been shown that bigger models are more data-efficient.
Some problems with the power law extrapolation for GDP:
The graph is for the whole world, not just the technological leading edge, which obscures the thing which is conceivably relevant (the endogenous trend in tech advancement at the leading edge)
The power law model is a bad fit for the GDP per capita of the first world in the last 50-100 years
Having built a toy endogenous model of economic growth, I see no gears-level reason to expect power law growth in our current regime. (Disclaimer: I’m not an economist, and haven’t tested my model on anything.) The toy model presented in the OpenPhil report is much simpler and IMO less realistic.
Agreed on points 1+2. On 3, depends on what you mean by current regime—seems like AI tech could totally lead to much faster growth than today, and in particular faster than exponential growth. Are you modelling history as a series of different regimes, each one of which is exponential but taken together comprise power-law growth?
seems like AI tech could totally lead to much faster growth than today, and in particular faster than exponential growth
Strongly agree.
Are you modelling history as a series of different regimes, each one of which is exponential but taken together comprise power-law growth?
I am not. The model is fully continuous, and involves the variables {science, technology, population, capital}. When you run the model, it naturally gives rise to a series of “phase changes”. The phase changes are smooth[1] but still quite distinct. Some of them are caused by changes in which inputs are bottlenecking a certain variable.
Super-exponential growth (Sci&Tech bottlenecked by labor surplus; ∞ BC to ~1700 AD (??))
Steady exponential growth
Fast exponential growth for a short period (population growth slows, causing less consumption)
Slow exponential growth for some time (less population growth --> less science --> less economic growth after a delay)
Super-exponential growth as AI replaces human researchers (population stops bottlenecking Sci&Tech as capital can be converted into intelligence)
My claim is that:
We are in phase 4, and that we don’t have enough automation of research to see the beginnings of phase 5 in GDP data.
Extrapolating GDP data tells us basically zero about when phase 5 will start. The timing can only be predicted with object-level reasoning about AI.
Phase 4 doesn’t fit the model of “growth always increases from one phase to the next”. Indeed, if you look at real economic data, the first world has had lower growth in recent decades than it did previously. Hence, power law extrapolation across phases is inappropriate.
As I think about this more and compare to what actually happened in history, I’m starting to doubt my model a lot more, since I’m not sure if the timing and details of the postulated phases line up properly with real world data.
I’d be interested to dig into this claim more. What exactly is the claim, and what is the justification for it? If the claim is something like “For most tasks, the thinking machines seem to need 0 to 3 orders of magnitude more experience on the task before they equal human performance” then I tentatively agree. But if it’s instead 6 to 9 OOMs, or even just a solid 3 OOMs, I’d say “citation needed!”
No precise claim, I’m afraid! The whole post was written from a place of “OK but what are my independent impressions on this stuff?”, and then setting down the things that felt most true in impression space. I guess I meant something like “IDK, seems like they maybe need 0 to 6 OOMs more”, but I just don’t think my impressions should be taken as strong evidence on this point.
The general point about the economic viability of automating specialized labour is about more than just data efficiency; there are other ~fixed costs for automating industries which mean small specialized industries will be later to be automated.
(It’s maybe worth commenting that the scenarios I describe here are mostly not like “current architecture just scales all the way to human-level and beyond with more compute”. If they actually do scale then maybe superhuman generalization happens significantly earlier in the process.)
Nice post! You’ve probably seen these already, but for the benefit of other readers, check out this awesome graph from the highly relevant OpenPhil report by Roodman:
More discussion of it here, blue lines added by me:
The red line is real historic GWP data; the splay of grey shades that continues it is the splay of possible futures calculated by the model. The median trajectory is the black line.
I messed around with a ruler to make some rough calculations, marking up the image with blue lines as I went. The big blue line indicates the point on the median trajectory where GWP is 10x what is was in 2019. Eyeballing it, it looks like it happens around 2040, give or take a year. The small vertical blue line indicates the year 2037. The small horizontal blue line indicates GWP in 2037 on the median trajectory.
Thus, it seems that between 2037 and 2040 on the median trajectory, GWP doubles. Thus TAI arrives around 2037 on the median trajectory.
...
Also: You say:
I’d be interested to dig into this claim more. What exactly is the claim, and what is the justification for it? If the claim is something like “For most tasks, the thinking machines seem to need 0 to 3 orders of magnitude more experience on the task before they equal human performance” then I tentatively agree. But if it’s instead 6 to 9 OOMs, or even just a solid 3 OOMs, I’d say “citation needed!”
Also, note that the current models are 3+ OOMs smaller than the human brain, and it’s been shown that bigger models are more data-efficient.
Some problems with the power law extrapolation for GDP:
The graph is for the whole world, not just the technological leading edge, which obscures the thing which is conceivably relevant (the endogenous trend in tech advancement at the leading edge)
The power law model is a bad fit for the GDP per capita of the first world in the last 50-100 years
Having built a toy endogenous model of economic growth, I see no gears-level reason to expect power law growth in our current regime. (Disclaimer: I’m not an economist, and haven’t tested my model on anything.) The toy model presented in the OpenPhil report is much simpler and IMO less realistic.
Agreed on points 1+2. On 3, depends on what you mean by current regime—seems like AI tech could totally lead to much faster growth than today, and in particular faster than exponential growth. Are you modelling history as a series of different regimes, each one of which is exponential but taken together comprise power-law growth?
Strongly agree.
I am not. The model is fully continuous, and involves the variables {science, technology, population, capital}. When you run the model, it naturally gives rise to a series of “phase changes”. The phase changes are smooth[1] but still quite distinct. Some of them are caused by changes in which inputs are bottlenecking a certain variable.
The phases predicted are:[2]
Super-exponential growth (Sci&Tech bottlenecked by labor surplus; ∞ BC to ~1700 AD (??))
Steady exponential growth
Fast exponential growth for a short period (population growth slows, causing less consumption)
Slow exponential growth for some time (less population growth --> less science --> less economic growth after a delay)
Super-exponential growth as AI replaces human researchers (population stops bottlenecking Sci&Tech as capital can be converted into intelligence)
My claim is that:
We are in phase 4, and that we don’t have enough automation of research to see the beginnings of phase 5 in GDP data.
Extrapolating GDP data tells us basically zero about when phase 5 will start. The timing can only be predicted with object-level reasoning about AI.
Phase 4 doesn’t fit the model of “growth always increases from one phase to the next”. Indeed, if you look at real economic data, the first world has had lower growth in recent decades than it did previously. Hence, power law extrapolation across phases is inappropriate.
I don’t mean this in a mathematically rigorous way
As I think about this more and compare to what actually happened in history, I’m starting to doubt my model a lot more, since I’m not sure if the timing and details of the postulated phases line up properly with real world data.
I’d be very interested to read more about the assumptions of your model, if there’s a write-up somewhere.
No precise claim, I’m afraid! The whole post was written from a place of “OK but what are my independent impressions on this stuff?”, and then setting down the things that felt most true in impression space. I guess I meant something like “IDK, seems like they maybe need 0 to 6 OOMs more”, but I just don’t think my impressions should be taken as strong evidence on this point.
The general point about the economic viability of automating specialized labour is about more than just data efficiency; there are other ~fixed costs for automating industries which mean small specialized industries will be later to be automated.
(It’s maybe worth commenting that the scenarios I describe here are mostly not like “current architecture just scales all the way to human-level and beyond with more compute”. If they actually do scale then maybe superhuman generalization happens significantly earlier in the process.)