Summary: a detailed look at offense/defense is required. Asymmetric attacks do exist.
Asymmetric Detail paragraph 1: The USA faced asymmetric warfare in the middle East and ultimately gave up. While almost all individual battles were won by US forces, the cost in terradollars was drastically skewed in favor of the attackers. 8 trillion USD over 20 years should be contrasted with the entire combined GDP of Iraq + Afghanistan, approximately 2.5 trillion over that time period. Some small portion of that was funding attacks (and foreign support)
Asymmetric Detail paragraph 2 : Where it gets really ridiculous is something like bioterrorism. Suppose, for the sake of argument, that Covid-19 really was developed through deliberate gain of function experiments. And it’s done in an expensive lab with no expenses spared. So 6.8 million/year. Suppose 5 years of experiments are done to find the most optimal pandemic causing organism before release. So 34 million dollars to develop the weapon, which did approximately 29.4 trillion in damage and killed approximately 3.4 million people. If the 60 lab staff went on mass shooting sprees, and killed 58 people each, they would have killed 3480 people. A bioweapon is approximately 1000 times as effective.
Note on cybersecurity: Most cyberattacks can be traced to:
(1) computing architectural decisions. The use of lower level languages and inherently insecure operating systems rather than containerized everything. Fundamentally insecure general purpose hardware, and untrusted code uses the same silicon as trusted code.
(2) mistakes in implementation. Perfect software on adequately good hardware cannot be hacked.
(3) channel insecurity. The way emails work, where the sender isn’t tied to a real identity and the status of the sender isn’t checked (corporate workers only want to hear from other corporate workers, jailers only from judges, bank officers only from the SEC and other bank workers) allows malware. Similarly, ‘fake news’ is only possible because it doesn’t require a publication license to open a website at all and there are minimal accountability standards for journalism.
Mitigations:
(1) is fixable, we’ll just need all new silicon, new languages, and new OSes. Some use of existing crypto ASICs and enclave CPUs is already done.
(2) is fixable, we just need to formally prove key parts of the software stack
(3) is fixable. Being a journalist has to be a title and it would be a licensed profession with requirements and standards. Calling yourself one without the license would be a crime. Opening a website that allegedly offers news would require a license and the web backend would need to contain some consensus mechanism to validate a given site. (blockchain might be used here). The same would be true for any other online service. Sending emails would similarly require verification and a connection to a business or a person.
Everything outside of this would be the dark web. Note that all the mitigations have costs, some substantial. There is less free speech in this world, less open communication, and computers are less efficient because of using dedicated silicon (CPU and memory) to handle separate programs.
Uses for AI Pauses: So while I personally think AI pauses are hopeless, I will note here that the above would be what you want to fix during one. You can’t find out the latest issues with more advanced AI than the current SOTA during a pause, but you could lock the world’s computers down to reduce the damage when an AI tries to escape.
Conclusion: Asymmetric attacks do exist, cyberattacks are an example where the advantage favors the defender, while bioattacks favor the attacker. With that said I do agree with the OPs general feeling that the best way to prepare for an upcoming age of AI and advanced weapons made possible by AI is to lock and load. You can’t really hope to compete in a world of advanced weapons by begging other countries not to build them.
Summary: a detailed look at offense/defense is required. Asymmetric attacks do exist.
Asymmetric Detail paragraph 1: The USA faced asymmetric warfare in the middle East and ultimately gave up. While almost all individual battles were won by US forces, the cost in terradollars was drastically skewed in favor of the attackers. 8 trillion USD over 20 years should be contrasted with the entire combined GDP of Iraq + Afghanistan, approximately 2.5 trillion over that time period. Some small portion of that was funding attacks (and foreign support)
Asymmetric Detail paragraph 2 : Where it gets really ridiculous is something like bioterrorism. Suppose, for the sake of argument, that Covid-19 really was developed through deliberate gain of function experiments. And it’s done in an expensive lab with no expenses spared. So 6.8 million/year. Suppose 5 years of experiments are done to find the most optimal pandemic causing organism before release. So 34 million dollars to develop the weapon, which did approximately 29.4 trillion in damage and killed approximately 3.4 million people. If the 60 lab staff went on mass shooting sprees, and killed 58 people each, they would have killed 3480 people. A bioweapon is approximately 1000 times as effective.
Note on cybersecurity: Most cyberattacks can be traced to:
(1) computing architectural decisions. The use of lower level languages and inherently insecure operating systems rather than containerized everything. Fundamentally insecure general purpose hardware, and untrusted code uses the same silicon as trusted code.
(2) mistakes in implementation. Perfect software on adequately good hardware cannot be hacked.
(3) channel insecurity. The way emails work, where the sender isn’t tied to a real identity and the status of the sender isn’t checked (corporate workers only want to hear from other corporate workers, jailers only from judges, bank officers only from the SEC and other bank workers) allows malware. Similarly, ‘fake news’ is only possible because it doesn’t require a publication license to open a website at all and there are minimal accountability standards for journalism.
Mitigations:
(1) is fixable, we’ll just need all new silicon, new languages, and new OSes. Some use of existing crypto ASICs and enclave CPUs is already done.
(2) is fixable, we just need to formally prove key parts of the software stack
(3) is fixable. Being a journalist has to be a title and it would be a licensed profession with requirements and standards. Calling yourself one without the license would be a crime. Opening a website that allegedly offers news would require a license and the web backend would need to contain some consensus mechanism to validate a given site. (blockchain might be used here). The same would be true for any other online service. Sending emails would similarly require verification and a connection to a business or a person.
Everything outside of this would be the dark web. Note that all the mitigations have costs, some substantial. There is less free speech in this world, less open communication, and computers are less efficient because of using dedicated silicon (CPU and memory) to handle separate programs.
Uses for AI Pauses: So while I personally think AI pauses are hopeless, I will note here that the above would be what you want to fix during one. You can’t find out the latest issues with more advanced AI than the current SOTA during a pause, but you could lock the world’s computers down to reduce the damage when an AI tries to escape.
Conclusion: Asymmetric attacks do exist, cyberattacks are an example where the advantage favors the defender, while bioattacks favor the attacker. With that said I do agree with the OPs general feeling that the best way to prepare for an upcoming age of AI and advanced weapons made possible by AI is to lock and load. You can’t really hope to compete in a world of advanced weapons by begging other countries not to build them.