I don’t really feel like anything you are saying undermines my position here, or defends the part of Eliezer’s picture I’m objecting to.
(ETA: but I agree with you that it’s the right kind of model to be talking about and is good to bring up explicitly in discussion. I think my failure to do so is mostly a failure of communication.)
I usually think about models that show the same kind of phase transition you discuss, though usually significantly more sophisticated models and moving from exponential to hyperbolic growth (you only get an exponential in your model because of the specific and somewhat implausible functional form for technology in your equation).
With humans alone I expect efficiency to double roughly every year based on the empirical returns curves, though it depends a lot on the trajectory of investment over the coming years. I’ve spent a long time thinking and talking with people about these issues.
At the point when the work is largely done by AI, I expect progress to be maybe 2x faster, so doubling every 6 months. And them from there I expect a roughly hyperbolic trajectory over successive doublings.
If takeoff is fast I still expect it to most likely be through a similar situation, where e.g. total human investment in AI R&D never grows above 1% and so at the time when takeoff occurs the AI companies are still only 1% of the economy.
1/(singularity_year–current_year). It’s the solution to the differential equation f′(x)=f(x)2 instead of f′(x)=f(x). I usually use it more broadly for 1/(singularity_year–current_year)α, which is the solution to f′(x)=f(x)1+1/α
Why do you use this form? Do you lean more on: 1. Historical trends that look hyperbolic; 2. Specific dynamical models like: let α be the synergy between “different innovations” as they’re producing more innovations; this gives f’(x) = f(x)^(1+α) *; or another such model?; 3. Something else?
I wonder if there’s a Paul-Eliezer crux here about plausible functional forms. For example, if Eliezer thinks that there’s very likely also a tech tree of innovations that change the synergy factor α, we get something like e.g. (a lower bound of) f’(x) = f(x)^f(x). IDK if there’s any help from specific forms; just that, it’s plausible that there’s forms that are (1) pretty simple, pretty straightforward lower bounds from simple (not necessarily high confidence) considerations of the dynamics of intelligence, and (2) look pretty similar to hyperbolic growth, until they don’t, and the transition happens quickly. Though maybe, if Eliezer thinks any of this and also thinks that these superhyperbolic synergy dynamics are already going on, and we instead use a stochastic differential equation, there should be something more to say about variance or something pre-End-times.
*ETA: for example, if every innovation combines with every other existing innovation to give one unit of progress per time, we get the hyperbolic f’(x) = f(x)^2; if innovations each give one progress per time but don’t combine, we get the exponential f’(x) = f(x).
I think there are two easy ways to get hyperbolic growth:
As long as there is free energy in the environment, without any technological change you can grow like f′(x)=f(x). Then if there is any technological progress that can be driven by your expanding physical civilization, then you get f′(x)=f(x)1+α, where α depends on how fast the returns to technology diminish.
Even without physical growth, if you have sufficiently good returns to technology (as we observe for historical technologies, if you treat doubling food as doubling output, or for modern information technology) then you end up with a similar functional form.
That would feel more like “plausible guess” if we didn’t have any historical data, but given that historical growth has in fact accelerated a huge amount it seems like a solid best guess to me. There’s been a bunch of debate about whether the historical data implies something kind of like this kind of functional form, or merely implies some kind of dramatic acceleration and is consistent with this functional form. But either way, it seems like the good bet is further dramatic acceleration if we either start returning energy capture to output (via AI) or start getting overall technological progress that is similar to existing rates of progress in computer hardware and software (via AI).
I don’t really feel like anything you are saying undermines my position here, or defends the part of Eliezer’s picture I’m objecting to.
(ETA: but I agree with you that it’s the right kind of model to be talking about and is good to bring up explicitly in discussion. I think my failure to do so is mostly a failure of communication.)
I usually think about models that show the same kind of phase transition you discuss, though usually significantly more sophisticated models and moving from exponential to hyperbolic growth (you only get an exponential in your model because of the specific and somewhat implausible functional form for technology in your equation).
With humans alone I expect efficiency to double roughly every year based on the empirical returns curves, though it depends a lot on the trajectory of investment over the coming years. I’ve spent a long time thinking and talking with people about these issues.
At the point when the work is largely done by AI, I expect progress to be maybe 2x faster, so doubling every 6 months. And them from there I expect a roughly hyperbolic trajectory over successive doublings.
If takeoff is fast I still expect it to most likely be through a similar situation, where e.g. total human investment in AI R&D never grows above 1% and so at the time when takeoff occurs the AI companies are still only 1% of the economy.
Excuse my ignorance, what does a hyperbolic function look like? If an exponential is f(x) = r^x, what is f(x) for a hyperbolic function?
1/(singularity_year–current_year). It’s the solution to the differential equation f′(x)=f(x)2 instead of f′(x)=f(x). I usually use it more broadly for 1/(singularity_year–current_year)α, which is the solution to f′(x)=f(x)1+1/α
Why do you use this form? Do you lean more on:
1. Historical trends that look hyperbolic;
2. Specific dynamical models like: let α be the synergy between “different innovations” as they’re producing more innovations; this gives f’(x) = f(x)^(1+α) *; or another such model?;
3. Something else?
I wonder if there’s a Paul-Eliezer crux here about plausible functional forms. For example, if Eliezer thinks that there’s very likely also a tech tree of innovations that change the synergy factor α, we get something like e.g. (a lower bound of) f’(x) = f(x)^f(x). IDK if there’s any help from specific forms; just that, it’s plausible that there’s forms that are (1) pretty simple, pretty straightforward lower bounds from simple (not necessarily high confidence) considerations of the dynamics of intelligence, and (2) look pretty similar to hyperbolic growth, until they don’t, and the transition happens quickly. Though maybe, if Eliezer thinks any of this and also thinks that these superhyperbolic synergy dynamics are already going on, and we instead use a stochastic differential equation, there should be something more to say about variance or something pre-End-times.
*ETA: for example, if every innovation combines with every other existing innovation to give one unit of progress per time, we get the hyperbolic f’(x) = f(x)^2; if innovations each give one progress per time but don’t combine, we get the exponential f’(x) = f(x).
I think there are two easy ways to get hyperbolic growth:
As long as there is free energy in the environment, without any technological change you can grow like f′(x)=f(x). Then if there is any technological progress that can be driven by your expanding physical civilization, then you get f′(x)=f(x)1+α, where α depends on how fast the returns to technology diminish.
Even without physical growth, if you have sufficiently good returns to technology (as we observe for historical technologies, if you treat doubling food as doubling output, or for modern information technology) then you end up with a similar functional form.
That would feel more like “plausible guess” if we didn’t have any historical data, but given that historical growth has in fact accelerated a huge amount it seems like a solid best guess to me. There’s been a bunch of debate about whether the historical data implies something kind of like this kind of functional form, or merely implies some kind of dramatic acceleration and is consistent with this functional form. But either way, it seems like the good bet is further dramatic acceleration if we either start returning energy capture to output (via AI) or start getting overall technological progress that is similar to existing rates of progress in computer hardware and software (via AI).
Nitpick: Isn’t 1/xα the solution for f′(x)=f(x)1+1α modulo constants? Or equivalently, 1x1α is the solution to f′(x)=f(x)1+α.
Yep, will fix.
-r/x
Finally a definitely of The Singularity that actually involves a mathematical singularity! Thank you.