As far as I can tell, their results find that bio experts don’t perform notably better than students at creating bioweapons. (Given just access to the internet.)
This seems somewhat surprising to me as I would have naively expected a large gap between experts and students.
This might indicate some issues with the tasks or the limited sample size. Or perhaps internet access and general research ability dominates over prior experience?
Minimally, it might be nice to verify that bioweapons experts perform significantly better than students on this evaluation to verify that there is some signal in the measurement. (I don’t see these results anywhere, though I haven’t read in detail.)
The main threat model with LLMs is that it gives amateurs expert-level biology knowledge. But this study indicates that expert level knowledge isn’t actually helpful, which implies we don’t need to worry about LLMs giving people expert-level biology knowledge.
Some alternative interpretations:
The study doesn’t accurately measure the gap between experts and non-expert
The knowledge needed to build a bioweapon is super niche. It’s not enough to be a biology PhD with wet lab experience; you have to specifically be an expert on gain of function research or something.
LLMs won’t help people build bioweapons by giving them access to special knowledge. Instead, they help them in other ways (e.g. by accelerating them or making them more competent as planners).
As far as I can tell, their results find that bio experts don’t perform notably better than students at creating bioweapons. (Given just access to the internet.)
This seems somewhat surprising to me as I would have naively expected a large gap between experts and students.
This might indicate some issues with the tasks or the limited sample size. Or perhaps internet access and general research ability dominates over prior experience?
Minimally, it might be nice to verify that bioweapons experts perform significantly better than students on this evaluation to verify that there is some signal in the measurement. (I don’t see these results anywhere, though I haven’t read in detail.)
This is surprising – thanks for bringing this up!
The main threat model with LLMs is that it gives amateurs expert-level biology knowledge. But this study indicates that expert level knowledge isn’t actually helpful, which implies we don’t need to worry about LLMs giving people expert-level biology knowledge.
Some alternative interpretations:
The study doesn’t accurately measure the gap between experts and non-expert
The knowledge needed to build a bioweapon is super niche. It’s not enough to be a biology PhD with wet lab experience; you have to specifically be an expert on gain of function research or something.
LLMs won’t help people build bioweapons by giving them access to special knowledge. Instead, they help them in other ways (e.g. by accelerating them or making them more competent as planners).