In fairness, “biosecurity” is perhaps the #2 longtermist cause area in effective-altruist circles. I’m not sure how much of the emphasis on this is secretly motivated by concerns about AI unleashing super-smallpox (or nanobots), versus motivated by the relatively normal worry that some malevolent group of ordinary humans might unleash super-smallpox. But regardless of motivation, I’d expect that almost all longtermist biosecurity work (which tends to be focused on worst-case GCBRs) is helpful for both human- and AI-induced scenarios.
It would be interesting to consider other potential “swiss cheese approach” attempts to patch humanity’s most vulnerable attack surfaces:
Trying to harden all countries’ nuclear weapons control systems against hacking and other manipulation attempts. (EA also does some work on nuclear risk, although here I think the kinds of work that EA focuses on, like ALLFED-style recovery after a war, might not be particularly helpful when it comes to AI-nuclear-risk in particular.)
Trying to “harvest the low-hanging fruit” by exhausting many of the easiest opportunities for an AI to make money online, so that most of the fruit is picked by the time a rouge AI comes along. Although picking the low-hanging fruit might be very destructive if it mostly involves, eg, committing crimes or scamming people out of their money. (For better or worse, I think we can expect private actors to be sufficiently motivated to do plenty of AI-assisted fruit-picking without needing encouragement from EA! Although smarter and smarter AI could probably reach higher and higher fruit, so you’ll never be able to truly get it all.)
Somehow trying to make the world resistant to super-persuasive ideological propaganda / bribery / scams / other forms of psychological manipulation? I don’t really see how we could defend against this possibility besides maybe taking the same “low-hanging fruit” approach. But I’d worry that a low-hanging fruit approach would be even more destructive in the “marketplace of ideas” than in the financial markets, making the world even more chaotic and crazy at exactly the wrong time.
One simpler attack surface that we could mitigate would be the raw availability of compute on earth—it would probably be pretty easy for the military of the USA, if they were so inclined, to draw up an attack plan for quickly destroying most of the world’s GPU datacenters and semiconductor fabs, using cruise missiles and the like. Obviously this would seriously ruffle diplomatic feathers and would create an instant worldwide economic crisis. But I’m guessing you might be able to quickly reduce the world’s total stock of compute by 1-2 orders of magnitude, which could be useful in a pinch. (Idk exactly how concentrated the world’s compute resources are.)
For a less violent, more long-term and incremental plan, it might be possible to work towards some kind of regulatory scheme whereby major governments maintained “kill switches” that could disable datacenters and fabs within their own borders, plus maybe had cyberattacks queued up to use on other countries’ datacenters and fabs. Analogous to how the NSA is able to monitor lots of the world’s internet traffic today, and how many nations might have kill switches for disabling/crippling the nation’s internet access in a pinch.
Other biosecurity work besides wet-lab restrictions, like creating “Super PPE” and other pathogen-agnostic countermeasures. This wouldn’t work against advanced nanotech, but it might be enough to foil cruder plans based on unleashing engineered pandemics.
Trying to identify other assorted choke-points that might come in handy in a pinch, such as disabling the world’s global positioning system satellites in order to instantly cripple lots of autonomous robotic vehicles/drones/etc.
Laying the groundwork for a “Vulnerable World Hypothesis”-style global surveillance state, although this is obviously a double-edged sword for many reasons.
Trying to promote even really insubstantial, token gestures of international cooperation on AI alignment, in the hopes that every little bit helps—I would love to see leading world powers come out with even a totally unenforceable, non-binding statement along the lines of “severely misaligned superintelligent AI cannot be contained and must never be built”. Analogous to various probably-insincere but nevertheless-somewhat-reassuring international statements that “nuclear war cannot be won and must never be fought”.
I agree with @shminux that these hacky patches would be worth little in the face of a truly superintelligent AI. So, eventually, the more central problems of alignment and safe deployment will have to be solved. But along the way, some of these approaches might help might buy crucial time on our way to solving the core problems—or at least help us die with a little more dignity.
In fairness, “biosecurity” is perhaps the #2 longtermist cause area in effective-altruist circles. I’m not sure how much of the emphasis on this is secretly motivated by concerns about AI unleashing super-smallpox (or nanobots), versus motivated by the relatively normal worry that some malevolent group of ordinary humans might unleash super-smallpox. But regardless of motivation, I’d expect that almost all longtermist biosecurity work (which tends to be focused on worst-case GCBRs) is helpful for both human- and AI-induced scenarios.
It would be interesting to consider other potential “swiss cheese approach” attempts to patch humanity’s most vulnerable attack surfaces:
Trying to harden all countries’ nuclear weapons control systems against hacking and other manipulation attempts. (EA also does some work on nuclear risk, although here I think the kinds of work that EA focuses on, like ALLFED-style recovery after a war, might not be particularly helpful when it comes to AI-nuclear-risk in particular.)
Trying to “harvest the low-hanging fruit” by exhausting many of the easiest opportunities for an AI to make money online, so that most of the fruit is picked by the time a rouge AI comes along. Although picking the low-hanging fruit might be very destructive if it mostly involves, eg, committing crimes or scamming people out of their money. (For better or worse, I think we can expect private actors to be sufficiently motivated to do plenty of AI-assisted fruit-picking without needing encouragement from EA! Although smarter and smarter AI could probably reach higher and higher fruit, so you’ll never be able to truly get it all.)
Somehow trying to make the world resistant to super-persuasive ideological propaganda / bribery / scams / other forms of psychological manipulation? I don’t really see how we could defend against this possibility besides maybe taking the same “low-hanging fruit” approach. But I’d worry that a low-hanging fruit approach would be even more destructive in the “marketplace of ideas” than in the financial markets, making the world even more chaotic and crazy at exactly the wrong time.
One simpler attack surface that we could mitigate would be the raw availability of compute on earth—it would probably be pretty easy for the military of the USA, if they were so inclined, to draw up an attack plan for quickly destroying most of the world’s GPU datacenters and semiconductor fabs, using cruise missiles and the like. Obviously this would seriously ruffle diplomatic feathers and would create an instant worldwide economic crisis. But I’m guessing you might be able to quickly reduce the world’s total stock of compute by 1-2 orders of magnitude, which could be useful in a pinch. (Idk exactly how concentrated the world’s compute resources are.)
For a less violent, more long-term and incremental plan, it might be possible to work towards some kind of regulatory scheme whereby major governments maintained “kill switches” that could disable datacenters and fabs within their own borders, plus maybe had cyberattacks queued up to use on other countries’ datacenters and fabs. Analogous to how the NSA is able to monitor lots of the world’s internet traffic today, and how many nations might have kill switches for disabling/crippling the nation’s internet access in a pinch.
Other biosecurity work besides wet-lab restrictions, like creating “Super PPE” and other pathogen-agnostic countermeasures. This wouldn’t work against advanced nanotech, but it might be enough to foil cruder plans based on unleashing engineered pandemics.
Trying to identify other assorted choke-points that might come in handy in a pinch, such as disabling the world’s global positioning system satellites in order to instantly cripple lots of autonomous robotic vehicles/drones/etc.
Laying the groundwork for a “Vulnerable World Hypothesis”-style global surveillance state, although this is obviously a double-edged sword for many reasons.
Trying to promote even really insubstantial, token gestures of international cooperation on AI alignment, in the hopes that every little bit helps—I would love to see leading world powers come out with even a totally unenforceable, non-binding statement along the lines of “severely misaligned superintelligent AI cannot be contained and must never be built”. Analogous to various probably-insincere but nevertheless-somewhat-reassuring international statements that “nuclear war cannot be won and must never be fought”.
I agree with @shminux that these hacky patches would be worth little in the face of a truly superintelligent AI. So, eventually, the more central problems of alignment and safe deployment will have to be solved. But along the way, some of these approaches might help might buy crucial time on our way to solving the core problems—or at least help us die with a little more dignity.