I recently stumbled upon this paper “The World’s Technological Capacity to Store, Communicate, and Compute Information”, which has some neat overviews regarding data storage, broadcasting and compute trends: From a quick look at the Figures my impression is that compute and storage look very much like ‘just’ exponential, while there is a super-exponential figure (Fig. 4) for the total communication bandwidth (1986-2007)[1]
General
I can see two mechanisms that potentially may make it less random.
That makes sense. Now that I think about this, I could well imagine that something like “scale is all you need” is sufficiently true that randomness doesn’t shift the expected date by a large amount.
if the data will predict for example singularity in 2030, it will likely mean AGI earlier than that.
Good point! I think that the time span around and before the first AGI will be most relevant to us as it probably provides the largest possibility to steer the outcome to something good, but this indeed is not the date we found get in a power-law singularity. This feels quite related to the discussion around biological anchors for estimating the necessary compute for transformative AI and the conclusions one can draw from this. I feel that if one thinks that these are informative, even ‘just’ the exponential compute trends provide rather strong bounds (at least compared to having biological time-scales or such as reference).
t is not very likely that the control parameter will be that easily measurable and will obey power-law that good. I think it is a very high risk—very high gain project (very high gain, because if the prediction will be very clear it will be possible to persuade more people that the problem is important).
Regarding persuading people: I am not sure whether such a trend would make such a large psychological difference compared to the things that we already have: All Possible Views About Humanity’s Future Are Wild. But it would still be a noteworthy finding in any case
Quick hand-wavy estimate whether the trend continued in the last 15 years: If I just assume ‘trend continues’ to mean ‘doubling time halves every 7 years with a factor x40 from 2000 to 2007’ (this isn’t power law, but much easier for me to think about and hopefully close enough in this parameter range) we’d have to find an increase in global bandwidth by (a factor of x40 is roughly 1.5 OOM and we get this factor twice in the first 7 years and 4 times in the second step which makes) roughly 9 orders of magnitude from 2007 to 2021. I did not try to find current numbers, but 9 OOM sound unlikely to me. At least my data usage did increase by maybe 3 to 5 OOM, but 9 OOM just seems too high. Thus, I anecdotally conclude that this trend very probably slowed down in the last 15 years compared to ‘doubling time halves every 7 years’. But to be fair, I would not have correctly predicted the 1986-2007 trend either—so this shouldn’t be taken too seriously.
Trends of different quantities:
Generally, I agree with your points :)
I recently stumbled upon this paper “The World’s Technological Capacity to Store, Communicate, and Compute Information”, which has some neat overviews regarding data storage, broadcasting and compute trends:
From a quick look at the Figures my impression is that compute and storage look very much like ‘just’ exponential, while there is a super-exponential figure (Fig. 4) for the total communication bandwidth (1986-2007)[1]
General
That makes sense. Now that I think about this, I could well imagine that something like “scale is all you need” is sufficiently true that randomness doesn’t shift the expected date by a large amount.
Good point! I think that the time span around and before the first AGI will be most relevant to us as it probably provides the largest possibility to steer the outcome to something good, but this indeed is not the date we found get in a power-law singularity.
This feels quite related to the discussion around biological anchors for estimating the necessary compute for transformative AI and the conclusions one can draw from this. I feel that if one thinks that these are informative, even ‘just’ the exponential compute trends provide rather strong bounds (at least compared to having biological time-scales or such as reference).
Regarding persuading people: I am not sure whether such a trend would make such a large psychological difference compared to the things that we already have: All Possible Views About Humanity’s Future Are Wild. But it would still be a noteworthy finding in any case
Quick hand-wavy estimate whether the trend continued in the last 15 years: If I just assume ‘trend continues’ to mean ‘doubling time halves every 7 years with a factor x40 from 2000 to 2007’ (this isn’t power law, but much easier for me to think about and hopefully close enough in this parameter range) we’d have to find an increase in global bandwidth by (a factor of x40 is roughly 1.5 OOM and we get this factor twice in the first 7 years and 4 times in the second step which makes) roughly 9 orders of magnitude from 2007 to 2021. I did not try to find current numbers, but 9 OOM sound unlikely to me. At least my data usage did increase by maybe 3 to 5 OOM, but 9 OOM just seems too high. Thus, I anecdotally conclude that this trend very probably slowed down in the last 15 years compared to ‘doubling time halves every 7 years’. But to be fair, I would not have correctly predicted the 1986-2007 trend either—so this shouldn’t be taken too seriously.