I appreciate you raising this point about whether LLMs alleviate the key bottlenecks on bioterrorism. I skimmed the paper you linked, thought about my previous evidence, and am happy to say that I’m much less certain that I was before.
My previous thinking for why I believe LLMs exacerbate biosecurity risks:
Kevin Esvelt said so on the 80k podcast, see also the small experiment he mentions. Okay evidence. (I have not listened to the entire episode)
Anthropic says so in this blog post. Upon reflection I think this is worse evidence than I previously thought (seems hard to imagine seeing the opposite conclusion from their research, given how vague the blog post is; access to their internal reports would help).
The Montague 2023 paper you link: the main bottleneck to high consequence biological attacks is actually R&D to create novel threats, especially spread testing which needs to take part in a realistic deployment environment. This requires both being purposeful and being having a bunch of resources, so policies need not be focused on decreasing democratic access to biotech and knowledge.
I don’t find the paper’s discussion of ‘why extensive spread testing is necessary’ to be super convincing, but it’s reasonable and I’m updating somewhat toward this position. That is, I’m not convinced either way. I would have a better idea if I knew how accurate a priori spread forecasting has been for bioweapons programs in the past.
I think the “LLMs exacerbate biosecurity risks” still goes through even if I mostly buy Montague’s arguments, given that those arguments are partially specific to high consequence attacks. Additionally, Montague is mainly arguing that democratization of biotech / info doesn’t help with the main barriers, not that it doesn’t help at all:
The reason spread-testing has not previously been perceived as a the defining stage of difficulty in the biorisk chain (see Figure 1), eclipsing all others, is that, until recently, the difficulties associated with the preceding steps in the risk chain were so high as to deter contemplation of the practical difficulties beyond them. With the advance of synthetic biology enabled by bioinformatic inferences on ’omics data, the perception of these prior barriers at earlier stages of the risk chain has receded.
So I think there exists some reasonable position (which likely doesn’t warrant focusing on LLM --> biosecurity risks, and is a bit of an epistemic cop out): LLMs increase biosecurity risks marginally even though they don’t affect the main blockers for bio weapon development.
Thanks again for your comment, I appreciate you pointing this out. This was one of the most valuable things I’ve read in the last 2 weeks.
Thanks for your comment!
I appreciate you raising this point about whether LLMs alleviate the key bottlenecks on bioterrorism. I skimmed the paper you linked, thought about my previous evidence, and am happy to say that I’m much less certain that I was before.
My previous thinking for why I believe LLMs exacerbate biosecurity risks:
Kevin Esvelt said so on the 80k podcast, see also the small experiment he mentions. Okay evidence. (I have not listened to the entire episode)
Anthropic says so in this blog post. Upon reflection I think this is worse evidence than I previously thought (seems hard to imagine seeing the opposite conclusion from their research, given how vague the blog post is; access to their internal reports would help).
The Montague 2023 paper you link: the main bottleneck to high consequence biological attacks is actually R&D to create novel threats, especially spread testing which needs to take part in a realistic deployment environment. This requires both being purposeful and being having a bunch of resources, so policies need not be focused on decreasing democratic access to biotech and knowledge.
I don’t find the paper’s discussion of ‘why extensive spread testing is necessary’ to be super convincing, but it’s reasonable and I’m updating somewhat toward this position. That is, I’m not convinced either way. I would have a better idea if I knew how accurate a priori spread forecasting has been for bioweapons programs in the past.
I think the “LLMs exacerbate biosecurity risks” still goes through even if I mostly buy Montague’s arguments, given that those arguments are partially specific to high consequence attacks. Additionally, Montague is mainly arguing that democratization of biotech / info doesn’t help with the main barriers, not that it doesn’t help at all:
So I think there exists some reasonable position (which likely doesn’t warrant focusing on LLM --> biosecurity risks, and is a bit of an epistemic cop out): LLMs increase biosecurity risks marginally even though they don’t affect the main blockers for bio weapon development.
Thanks again for your comment, I appreciate you pointing this out. This was one of the most valuable things I’ve read in the last 2 weeks.