Fantastic snapshot. I wonder (and worry) whether we’ll look back on it with similar feelings as those we have for What 2026 looks like now.
There is also no “last resort war plan” in which the president could break all of the unstable coordination failures and steer the ship.
[...]
There are no clear plans for what to do under most conditions, e.g. there is no clear plan for when and how the military should assume control over this technology.
These sound intuitively unlikely to me, by analogy to nuclear or bio. Of course, that is not to say these protocols will be sufficient or even sane, by analogy to nuclear or bio.
This makes it really unclear what to work on.
It’s not super obvious to me that there won’t be clever ways to change local incentives / improve coordination, and successful interventions in this direction would seem incredibly high-leveraged, since they’re upstream of many of the messy and decentralized failure modes. If they do exist, they probably look not like “a simple cooridnation mechanism”, and more like “a particular actor gradually steering high-stakes conversations (through a sequence of clever actions) to bootstrap minimal agreements”. Of course, similarity to past geopolitical situations does make it seem unlikely on priors.
There is no time to get to very low-risk worlds anymore. There is only space for risk reduction along the way.
My gut has been in agreement for some time that the most cost-effective x-risk reduction now probably looks like this.
Fantastic snapshot. I wonder (and worry) whether we’ll look back on it with similar feelings as those we have for What 2026 looks like now.
These sound intuitively unlikely to me, by analogy to nuclear or bio. Of course, that is not to say these protocols will be sufficient or even sane, by analogy to nuclear or bio.
It’s not super obvious to me that there won’t be clever ways to change local incentives / improve coordination, and successful interventions in this direction would seem incredibly high-leveraged, since they’re upstream of many of the messy and decentralized failure modes. If they do exist, they probably look not like “a simple cooridnation mechanism”, and more like “a particular actor gradually steering high-stakes conversations (through a sequence of clever actions) to bootstrap minimal agreements”. Of course, similarity to past geopolitical situations does make it seem unlikely on priors.
My gut has been in agreement for some time that the most cost-effective x-risk reduction now probably looks like this.