It seems very odd to have a discussion of arms race dynamics that is purely theoretical exploration of possible payoff matrices, and does not include a historically informed discussion of what seems like the obviously most analogous case, namely nuclear weapons research during the Second World War.
US nuclear researchers famously (IIRC, pls correct me if wrong!) thought there was a nontrivial chance their research would lead to human extinction, not just because nuclear war might do so but because e.g. a nuclear test explosion might ignite the atmosphere. They forged ahead anyway on the theory that otherwise the Nazis were going to get there first, and if they got there first they would use that advantage to lock in Nazi hegemony, and that was so bad an outcome it was worth a significant risk of human extinction to avoid.
Was that the wrong thing for them to have done under the circumstances? If so, why, and what can we say confidently in hindsight that should they have done instead? If not, why is the present situation saliently different? If China gets to AGI first that plausibly locks in CCP hegemony which is arguably similarly bad to locking in Nazi hegemony. Trying to convince the CCP that they will just kill themselves too if they do this, so they shouldn’t try, seems about as tractable as persuading Werner Heisenberg and his superiors during WWII that they shouldn’t try to build nukes because they might ignite the atmosphere.
It seems very odd to have a discussion of arms race dynamics that is purely theoretical exploration of possible payoff matrices, and does not include a historically informed discussion of what seems like the obviously most analogous case, namely nuclear weapons research during the Second World War.
US nuclear researchers famously (IIRC, pls correct me if wrong!) thought there was a nontrivial chance their research would lead to human extinction, not just because nuclear war might do so but because e.g. a nuclear test explosion might ignite the atmosphere. They forged ahead anyway on the theory that otherwise the Nazis were going to get there first, and if they got there first they would use that advantage to lock in Nazi hegemony, and that was so bad an outcome it was worth a significant risk of human extinction to avoid.
Was that the wrong thing for them to have done under the circumstances? If so, why, and what can we say confidently in hindsight that should they have done instead? If not, why is the present situation saliently different? If China gets to AGI first that plausibly locks in CCP hegemony which is arguably similarly bad to locking in Nazi hegemony. Trying to convince the CCP that they will just kill themselves too if they do this, so they shouldn’t try, seems about as tractable as persuading Werner Heisenberg and his superiors during WWII that they shouldn’t try to build nukes because they might ignite the atmosphere.