The other issue is that falls out of this is that if anyone does successfully defect in secret, while every other power honors the ban, they get an insurmountable advantage. Self replicating factories, buried deep underground or under the ocean could give the side that does this a certain victory and control of the planet. Nukes won’t be enough etc, you can’t deal with an exponential problem with a linear amount of weapons. (once there are more self replicating nodes than the number of nukes on the planet, victory for the side with the AGI is probably certain. Conventional military would not be able to deal with swarm attacks, perfect aim and inter machine coordination and so on)
Victory comes for whichever side has control/monitoring over the majority of the planet, including the oceans, first. I don’t know of any technology that can achieve this that doesn’t first require you to have a controllable network of systems with the capabilities of AGI/ASI.
Controlling the GPUs isn’t enough, it’s too easy to build similar devices. Economics has concentrated most of the fabs in TSMC because it’s more efficient, but in a world where we know AGI is possible and GPU/TPUs are a strategic advantage, you would expect every world power to start building it’s own.
Yes, if diplomacy fails and it does come to an uncontrollable state and chaos/dangerous-hidden-compute/violence is the likely outcome… In the long run the exponential wins, but existing military power starts with an advantage. So a decisive early strike can abort early-stage runaway RSI-ing AGI. I really hope it won’t come to that. I really hope that more of a ‘worldwide monitoring and policing’ action will be adequate to prevent defection.
The other issue is that falls out of this is that if anyone does successfully defect in secret, while every other power honors the ban, they get an insurmountable advantage. Self replicating factories, buried deep underground or under the ocean could give the side that does this a certain victory and control of the planet. Nukes won’t be enough etc, you can’t deal with an exponential problem with a linear amount of weapons. (once there are more self replicating nodes than the number of nukes on the planet, victory for the side with the AGI is probably certain. Conventional military would not be able to deal with swarm attacks, perfect aim and inter machine coordination and so on)
Victory comes for whichever side has control/monitoring over the majority of the planet, including the oceans, first. I don’t know of any technology that can achieve this that doesn’t first require you to have a controllable network of systems with the capabilities of AGI/ASI.
Controlling the GPUs isn’t enough, it’s too easy to build similar devices. Economics has concentrated most of the fabs in TSMC because it’s more efficient, but in a world where we know AGI is possible and GPU/TPUs are a strategic advantage, you would expect every world power to start building it’s own.
Yes, if diplomacy fails and it does come to an uncontrollable state and chaos/dangerous-hidden-compute/violence is the likely outcome… In the long run the exponential wins, but existing military power starts with an advantage. So a decisive early strike can abort early-stage runaway RSI-ing AGI. I really hope it won’t come to that. I really hope that more of a ‘worldwide monitoring and policing’ action will be adequate to prevent defection.
Currently, the US military has good enough satellite monitoring of the world to detect most large scale engineering projects like undersea datacenter construction. This isn’t easy, undersea datacenter construction is easier than it sounds… Example: https://news.microsoft.com/source/features/sustainability/project-natick-underwater-datacenter/