In this reaction to Critch’s podcast, I wrote about some reasons to think that a singleton would be preferable to a multipolar scenario. Here’s another rather exotic argument.
[The dark forest theory] is explained very well near the end of the science fiction novel, The Dark Forest by Liu Cixin.
...
When two [interstellar] civilizations meet, they will want to know if the other is going to be friendly or hostile. One side might act friendly, but the other side won’t know if they are just faking it to put them at ease while armies are built in secret. This is called chains of suspicion. You don’t know for sure what the other side’s intentions are. On Earth this is resolved through communication and diplomacy. But for civilizations in different solar systems, that’s not possible due to the vast distances and time between message sent and received. Bottom line is, every civilization could be a threat and it’s impossible to know for sure, therefore they must be destroyed to ensure your survival.
Secure second strike is the ability to retaliate with your own nuclear strike if someone hits you with nukes. Secure second strike underpins mutually assured destruction. If nuclear war had a “first mover advantage”, where whoever launches nukes first wins because the country that is hit with nukes is unable to retaliate, that would be much worse for a game theory perspective, because there’s an incentive to be the first mover and launch a nuclear war (especially if you think your opponent might do the same).
My understanding is that the invention of nuclear submarines was helpful for secure second strike. There is so much ocean for them to hide in that it’s difficult to track and eliminate all of your opponent’s nuclear submarines and ensure they won’t be able to hit you back.
However, in Allan Dafoe’s article AI Governance: Opportunity and Theory of Impact, he mentions that AI processing of undersea sensors could increase the risk of nuclear war (presumably because it makes it harder for nuclear submarines to hide).
Point being, we don’t know what the game theory of a post-AGI world looks like. And we really don’t know what interstellar game theory between different AGIs looks like. (“A colonized solar system is plausibly a place where predators can see most any civilized activities of any substantial magnitude, and get to them easily if not quickly.”—source.) It might be that the best strategy is for multipolar AIs to unify into a singleton anyway.
Potential counterargument: Second-strike capabilities are still relevant in the interstellar setting. You could build a bunch of hidden ships in the oort cloud to ram the foe and do equal devastation if the other party does it first, deterring a first strike even with tensions and an absence of communication. Further, while the “ram with high-relativistic objects” idea works pretty well for preemptively ending a civilization confined to a handful of planets, AI’s would be able to colonize a bunch of little asteroids and KBO’s and comets in the oort cloud, and the higher level of dispersal would lead to preemptive total elimination being less viable.
In this reaction to Critch’s podcast, I wrote about some reasons to think that a singleton would be preferable to a multipolar scenario. Here’s another rather exotic argument.
Source. (Emphasis mine.)
Secure second strike is the ability to retaliate with your own nuclear strike if someone hits you with nukes. Secure second strike underpins mutually assured destruction. If nuclear war had a “first mover advantage”, where whoever launches nukes first wins because the country that is hit with nukes is unable to retaliate, that would be much worse for a game theory perspective, because there’s an incentive to be the first mover and launch a nuclear war (especially if you think your opponent might do the same).
My understanding is that the invention of nuclear submarines was helpful for secure second strike. There is so much ocean for them to hide in that it’s difficult to track and eliminate all of your opponent’s nuclear submarines and ensure they won’t be able to hit you back.
However, in Allan Dafoe’s article AI Governance: Opportunity and Theory of Impact, he mentions that AI processing of undersea sensors could increase the risk of nuclear war (presumably because it makes it harder for nuclear submarines to hide).
Point being, we don’t know what the game theory of a post-AGI world looks like. And we really don’t know what interstellar game theory between different AGIs looks like. (“A colonized solar system is plausibly a place where predators can see most any civilized activities of any substantial magnitude, and get to them easily if not quickly.”—source.) It might be that the best strategy is for multipolar AIs to unify into a singleton anyway.
Potential counterargument: Second-strike capabilities are still relevant in the interstellar setting. You could build a bunch of hidden ships in the oort cloud to ram the foe and do equal devastation if the other party does it first, deterring a first strike even with tensions and an absence of communication. Further, while the “ram with high-relativistic objects” idea works pretty well for preemptively ending a civilization confined to a handful of planets, AI’s would be able to colonize a bunch of little asteroids and KBO’s and comets in the oort cloud, and the higher level of dispersal would lead to preemptive total elimination being less viable.
That’s possible, but I’m guessing that it’s not hard for a superintelligent AI to suddenly swallow an entire system using something like gray goo.