I’m pretty sure we have the computing power to run at least one AGI, but I can’t prove that. Still, restricting general computing progress should delay the arrival of AGI because the more hardware you have, the “dumber” you can be with solving the AGI software problem. (You can run less efficient algorithms.)
Global technological stasis seems just as hard or maybe harder than restricting AGI research. The incentives for AGI are huge, but there might be some points at which you have to spend a lot of money before you get much additional economic/political/military advantage. But when it comes to general computing progress, then it looks to be always the case that a bit more investment can always yield better returns, e.g. by squeezing more processor cores into a box a little more tightly.
Other difficulties of global technological stasis are discussed in the final chapter of GCR, called “The Totalitarian Threat.” Basically, you’d need some kind of world government, because any country that decides to slow down its computing progress rapidly falls behind other nations. But political progress is so slow that it seems unlikely we’ll get a world government in the next century unless somebody gets a decisive technological advantage via AGI, in which case we’re talking about the AGI problem anyway. (The world government scenario looks like the most plausible of Bostrom’s “crunches”, which is why I chose to discuss it.)
Global technological stasis seems just as hard or maybe harder than restricting AGI research.
I don’t want to link directly to what I believe is a solution to this problem, as it was so widely misinterpreted, but I thought there was already a solution to this problem that hinged on how expensive it is to harden fabrication plants. In that case, you wouldn’t even need one world government; a sufficiently large collaboration (e.g., NATO) would do the trick.
A world government may not be the only form of stable social equilibrium that could permanently thwart progress. Many regions of the world today have great difficulty building institutions that can support high growth. And historically, there are many places where progress stood still or retreated for significant periods of time. Economic and technological progress may not be as inevitable as is appears to us.
It seems to me like saying world government is necessary underestimates the potential impact of growth-undermining ideas. If all large governments for example buy into the idea that cutting government infrastructure spending in a recession boosts employment, then we can assume that global growth will slow as a result of the acceptance of this false assertion. To me the key seems less to have some world nabob pronouncing edicts on technology and more shifting the global economy so as to make edicts merely the gift wrap.
I will definitely take a look at that chapter of GCR. Thanks also for the other links. The little paper by Berglas I found interesting. Mr. K needs no comment, naturally. Might be good reading for similar reasons that Mien Kampf is. With De Garis I have long felt like the kook-factor was too high to warrant messing with. Anyone read him much? Maybe it would be good just for some ideas.
It certainly could be true that economic growth and technological progress can slow down. In fact, I suspect the former if not the latter will slow down, and perhaps the latter, too. That’s very different from stopping technological progress that will lead to AGI, though.
Not only can economic growth and technological progress slow down. They can stop and reverse. Just because we’re now further out in front than humanity has ever been before in history does not mean that we can’t go backwards. Economic growth is probably more likely to reverse than technological progress. That’s what a depression is, after all.
But a sufficiently bad global catastrophe, perhaps one that destroyed the electrical grid and other key infrastructure, could reverse a lot of technological progress too and perhaps knock us way back without necessarily causing complete extinction.
I think technological stasis could really use more discussion. For example I was able to find this paper by James Hughes discussing relinquishment. He however treats the issue as one of regulation and verification, similar to nuclear weapons, noting that:
We do not yet have effective global institutions capable of preventing determined research of any kind, even apocalyptic. I believe we will be build strong global institutions in the next couple of decades. Global institutions may regulate for safety, prevent weaponization, support technology transfer, and so on. But there will be no support for global governance that attempts to deny developing countries the right to emerging technologies.
Regulation and verification may indeed be a kind of Gordian knot. The more specific the technologies you want to stop, the harder it becomes to do that and still advance generally. Berglas recognizes that problem in his paper and so proposes restricting computing as a whole. Even this, however, may be too specific to cut the knot. The feasibility of stopping economic growth entirely so we never reach the point where regulation is necessary seems to me an unexplored question. If we look at global GDP growth over the past 50 years it’s been uniformly positive except for the most recent recession. It’s also been quite variable over short periods. Clearly stopping it for longer than a few years would require some new phenomenon driving a qualitative break with the past. That does not mean however that a stop is impossible.
There does exist a small minority camp within the economics profession advocating no-growth policies for environmental or similar reasons. I wonder if anyone has created a roadmap for bringing such policies about on a global level.
I’m pretty sure we have the computing power to run at least one AGI, but I can’t prove that. Still, restricting general computing progress should delay the arrival of AGI because the more hardware you have, the “dumber” you can be with solving the AGI software problem. (You can run less efficient algorithms.)
Global technological stasis seems just as hard or maybe harder than restricting AGI research. The incentives for AGI are huge, but there might be some points at which you have to spend a lot of money before you get much additional economic/political/military advantage. But when it comes to general computing progress, then it looks to be always the case that a bit more investment can always yield better returns, e.g. by squeezing more processor cores into a box a little more tightly.
Other difficulties of global technological stasis are discussed in the final chapter of GCR, called “The Totalitarian Threat.” Basically, you’d need some kind of world government, because any country that decides to slow down its computing progress rapidly falls behind other nations. But political progress is so slow that it seems unlikely we’ll get a world government in the next century unless somebody gets a decisive technological advantage via AGI, in which case we’re talking about the AGI problem anyway. (The world government scenario looks like the most plausible of Bostrom’s “crunches”, which is why I chose to discuss it.)
Relinquishment is also discussed (in very different ways) by Berglas (2009), Kaczynski (1995), and De Garis (2005).
I don’t want to link directly to what I believe is a solution to this problem, as it was so widely misinterpreted, but I thought there was already a solution to this problem that hinged on how expensive it is to harden fabrication plants. In that case, you wouldn’t even need one world government; a sufficiently large collaboration (e.g., NATO) would do the trick.
Quoting the B-man:
A world government may not be the only form of stable social equilibrium that could permanently thwart progress. Many regions of the world today have great difficulty building institutions that can support high growth. And historically, there are many places where progress stood still or retreated for significant periods of time. Economic and technological progress may not be as inevitable as is appears to us.
It seems to me like saying world government is necessary underestimates the potential impact of growth-undermining ideas. If all large governments for example buy into the idea that cutting government infrastructure spending in a recession boosts employment, then we can assume that global growth will slow as a result of the acceptance of this false assertion. To me the key seems less to have some world nabob pronouncing edicts on technology and more shifting the global economy so as to make edicts merely the gift wrap.
I will definitely take a look at that chapter of GCR. Thanks also for the other links. The little paper by Berglas I found interesting. Mr. K needs no comment, naturally. Might be good reading for similar reasons that Mien Kampf is. With De Garis I have long felt like the kook-factor was too high to warrant messing with. Anyone read him much? Maybe it would be good just for some ideas.
It certainly could be true that economic growth and technological progress can slow down. In fact, I suspect the former if not the latter will slow down, and perhaps the latter, too. That’s very different from stopping technological progress that will lead to AGI, though.
Not only can economic growth and technological progress slow down. They can stop and reverse. Just because we’re now further out in front than humanity has ever been before in history does not mean that we can’t go backwards. Economic growth is probably more likely to reverse than technological progress. That’s what a depression is, after all.
But a sufficiently bad global catastrophe, perhaps one that destroyed the electrical grid and other key infrastructure, could reverse a lot of technological progress too and perhaps knock us way back without necessarily causing complete extinction.
I think technological stasis could really use more discussion. For example I was able to find this paper by James Hughes discussing relinquishment. He however treats the issue as one of regulation and verification, similar to nuclear weapons, noting that:
Regulation and verification may indeed be a kind of Gordian knot. The more specific the technologies you want to stop, the harder it becomes to do that and still advance generally. Berglas recognizes that problem in his paper and so proposes restricting computing as a whole. Even this, however, may be too specific to cut the knot. The feasibility of stopping economic growth entirely so we never reach the point where regulation is necessary seems to me an unexplored question. If we look at global GDP growth over the past 50 years it’s been uniformly positive except for the most recent recession. It’s also been quite variable over short periods. Clearly stopping it for longer than a few years would require some new phenomenon driving a qualitative break with the past. That does not mean however that a stop is impossible.
There does exist a small minority camp within the economics profession advocating no-growth policies for environmental or similar reasons. I wonder if anyone has created a roadmap for bringing such policies about on a global level.
Out of curiosity, have you read my little essay “Slowing Moore’s Law”? It seems relevant.