I agree that a control group is vital for good science. Nonetheless, I think that such an experiment is valuable and informative, even if it doesn’t meet the high standards required by many professional science disciplines.
I believe in the necessity of acting under uncertainty. Even with its flaws, this study is sufficient evidence for us to want to enact temporary regulation at the same time as we work to provide more robust evaluations.
But… this study doesn’t provide evidence that LLMs increase bioweapon risk.
Thanks for highlighting the issue with the discourse here. People use the word evidence in two different ways which often results in people talking past one another.
I’m using your broader definition, where I imagine that Stella is excluding things that don’t meet a more stringent standard.
And my claim is that reasoning under uncertainty sometimes means making decisions based on evidence[1] that is weaker than we’d like.
But… this study doesn’t provide evidence that LLMs increase bioweapon risk.
Define evidence.
I’m not asking this just to be pedantic, but because I think it’ll make the answer to your objection clearer.
Evidence for X is when you see something that’s more likely in a world with X than in a world with some other condition not X.
Generally substantially more likely; for good reason many people only use “evidence” to mean “reasonably strong evidence.”
Thanks for highlighting the issue with the discourse here. People use the word evidence in two different ways which often results in people talking past one another.
I’m using your broader definition, where I imagine that Stella is excluding things that don’t meet a more stringent standard.
And my claim is that reasoning under uncertainty sometimes means making decisions based on evidence[1] that is weaker than we’d like.
Broad definition