It seems that in a counterfactual world where the DL revolution happened later and the DL era was compressed into a shorter timespan, our chances of alignment would be worse since that world’s equivalent of Redwood Research would have less time to do their research.
It seems to me that counterfactually changing the date of the start of the Deep Learning revolution has two impacts: it shortens or lengthens the Deep Learning era, and it accelerates or decelerates the arrival of AGI.
ie If you could have magically gotten Deep Learning to happen earlier, we would have had longer time in the DL era, because there would be more time where people are using the DL paradigm while there was less compute to do it with, and more time for us to learn more about how Deep Learning works. But also, it means there are more researcher-hours going into finding DL techniques, which overall probably speeds up AGI arrival times.
It seems like a (the?) crux here is which of these impacts predominates. How much additional safety progress do you get from marginal knowledge of AI paradigms, vs. how much additional safety progress do you get from additional years to work on the problem.
Making up some numbers: would we prefer to have another 10 years to work on the problem, in which it is only in the final 2 that we get to see the paradigm in which AGI will be built? Or would we prefer to have 6 years to work on the problem, during all of which we have access to the paradigm that will build AGI?
Suppose you could build an AGI in 1999 or 2009, but the AGI required a specialized, expensive supercomputer to run, and there was only 1-2 of such supercomputers in the world. Also suppose (for the sake of argument) that the AGI couldn’t create a botnet of itself using PCs or conventional servers, or that creating such a botnet would not significantly improve the AGI’s abilities (<2x improvement). Would that be a better outcome than an AGI that arrives in 2029 and can run on dozens or billions of machines which exist at that time?
Not having a hardware overhang makes your planet much safer. But it depends on how quickly researchers would develop methods for scaling AGI systems, either by building more supercomputers, or generalizing our code to run on more conventional machines. If this process takes years or decades we get to experiment with AGI in a relatively safe way. But if this step takes months, then I think the world ends in ~ 2000 or ~ 2010 (depending on our AGI arrival date).
It seems to me that counterfactually changing the date of the start of the Deep Learning revolution has two impacts: it shortens or lengthens the Deep Learning era, and it accelerates or decelerates the arrival of AGI.
ie If you could have magically gotten Deep Learning to happen earlier, we would have had longer time in the DL era, because there would be more time where people are using the DL paradigm while there was less compute to do it with, and more time for us to learn more about how Deep Learning works. But also, it means there are more researcher-hours going into finding DL techniques, which overall probably speeds up AGI arrival times.
It seems like a (the?) crux here is which of these impacts predominates. How much additional safety progress do you get from marginal knowledge of AI paradigms, vs. how much additional safety progress do you get from additional years to work on the problem.
Making up some numbers: would we prefer to have another 10 years to work on the problem, in which it is only in the final 2 that we get to see the paradigm in which AGI will be built? Or would we prefer to have 6 years to work on the problem, during all of which we have access to the paradigm that will build AGI?
Suppose you could build an AGI in 1999 or 2009, but the AGI required a specialized, expensive supercomputer to run, and there was only 1-2 of such supercomputers in the world. Also suppose (for the sake of argument) that the AGI couldn’t create a botnet of itself using PCs or conventional servers, or that creating such a botnet would not significantly improve the AGI’s abilities (<2x improvement). Would that be a better outcome than an AGI that arrives in 2029 and can run on dozens or billions of machines which exist at that time?
Maybe?
Not having a hardware overhang makes your planet much safer. But it depends on how quickly researchers would develop methods for scaling AGI systems, either by building more supercomputers, or generalizing our code to run on more conventional machines. If this process takes years or decades we get to experiment with AGI in a relatively safe way. But if this step takes months, then I think the world ends in ~ 2000 or ~ 2010 (depending on our AGI arrival date).