Like, we could extrapolate out 3 OOMs of compute/$ per decade to get an upper bound: very probably AGI before 2150-ish, if Moore’s law continues.
Projecting Moore’s Law to continue for 130 years more is almost surely incorrect. An upper bound that is conditional on that happening seems devoid of any actual predictive power. If we approach that level of computational power prior to AGI, it will almost surely be through some other mechanism than Moore’s Law, and so would be arbitrarily detached from that timeline.
Well Eliezer did explicitly state that “it was, predictably, a directional overestimate”. His concern was that it is a useless estimate, not that it didn’t roughly bound the amount of computation required.
Projecting Moore’s Law to continue for 130 years more is almost surely incorrect. An upper bound that is conditional on that happening seems devoid of any actual predictive power. If we approach that level of computational power prior to AGI, it will almost surely be through some other mechanism than Moore’s Law, and so would be arbitrarily detached from that timeline.
Seems right, IDK. But still, that’s a different kind of uncertainty than uncertainty about, like, the shape of algorithm-space.
Well Eliezer did explicitly state that “it was, predictably, a directional overestimate”. His concern was that it is a useless estimate, not that it didn’t roughly bound the amount of computation required.