I strong downvoted and strong disagree voted. The reason I did both is because I think what you’re describing is a genuinely insane standard to take for liability. Holding organizations liable for any action they take which they do not prove is safe is an absolutely terrible idea. It would either introduce enormous costs for doing anything, or allow anyone to be sued for anything they’ve previously done.
Holding organizations liable for any action they take which they do not prove is safe is an absolutely terrible idea
I think you’re talking past each other. I interpret Nathan as saying that he could prove that everyone on earth has been harmed, but that he couldn’t do that in a safe manner.
Thanks Quintin. That’s useful. I think the general standard of holding organizations liable for any action which they do not prove to be safe is indeed a terrible idea. I do think that certain actions may carry higher implicit harms, and should be held to a higher standard of caution.
Perhaps you, or others, will give me your opinions on the following list of actions. Where is a good point, in your opinion, to ‘draw the line’? Starting from what I would consider ‘highly dangerous and much worse than Llama2’ and going down to ‘less dangerous than Llama2’, here are some related actions.
Releasing for free a software product onto the internet explicitly engineered to help create bio-weapons. Advertising this product as containing not only necessary gene sequences and lab protocols, but also explicit instructions for avoiding government and organization safety screens. Advertising that this product shows multiple ways to create your desired bio-weapon, including using your own lab equipment or deceiving Contract Research Organizations into unwittingly assisting you.
Releasing the same software product with the same information, but not mentioning to anyone what it is intended for. Because it is a software product, rather than an ML model, the information is all correct and not mixed in with hallucinations. The user only needs to navigate to the appropriate part of the app to get the information, rather than querying a language model.
Releasing a software product not actively intended for the above purposes, but which does happen contain all that information and can incidentally be used for those purposes.
Releasing an LLM which was trained on a dataset containing this information, and can regurgitate the information accurately. Furthermore, tuning this LLM before release to be happy to help users with requests to carry out a bio-weapons project, and double-checking to make sure that the information given is accurate.
Releasing an LLM that happens to contain such information and could be used for such purposes, but the information is not readily available (e.g. requires jail breaking or fine tuning) and is contaminated with hallucinations which can make it difficult to know which portion of the information to trust.
Releasing an LLM that could be used for such purposes, but that doesn’t yet contain the information. Bad actors would have to acquire the relevant information and fine-tune the model on it in order for the model to be able to effectively assist them in making a weapon of mass destruction.
All of the below is my current personal opinion, and not intended to be normative.
Once you remove the “Advertising” part from (1), you’re left with “Releasing for free a software product onto the internet explicitly engineered to help create bio-weapons”, which means that (2) seems definitely worthy of criminal liability.
(3) turns that into “Releasing for free a software product onto the internet which contains information that may help create bio-weapons”, and is probably about where my fuzzy line for liability comes in. Some cases of (3) should be liable, some should not. If the information in it is just swept up as part of a big pile of publically available information, then no. If it contains substantial portions of non-public information about bio-weapon development, even as part of a much larger corpus of information not normally available to the public, then probably yes.
However: (4) re-introduces intent and should be criminally liable, since in this scenario it seems that the developer will happily fine-tunespecifically for compliance and accuracy of bio-weapon development information. If they just fine-tune for compliance with user requests in general and do some random sampling of accuracy of information across a broad range of requests that don’t specifically include bio-weapons development, we’re back to the (3) case where they should be liable in some cases but not others.
(5) and (6) seem no worse than Google and Wikipedia and should basically never be liable.
Also given the rest of the replies, I think he means that it would be challenging for a plaintiff to safely prove that Llama 2 enables terrorists to make bioweapons, not that the alleged harm is “making open-source AI without proving it safe” or such.
Thanks Daniel, yes. To be more clear: I have evidence that I do not feel comfortable presenting which I believe would be more convincing than the evidence I do feel comfortable presenting. I am working on finding more convincing evidence which is still safe to present. I am seeking to learn what critics would consider to be cruxy evidence which might lie in the ‘safe for public discussion’ zone.
FWIW the heavy negative karma means that his answer is hidden by default, so that readers can’t easily see why OP thinks it might make sense to bring a lawsuit against Meta, which seems bad.
I strong downvoted and strong disagree voted. The reason I did both is because I think what you’re describing is a genuinely insane standard to take for liability. Holding organizations liable for any action they take which they do not prove is safe is an absolutely terrible idea. It would either introduce enormous costs for doing anything, or allow anyone to be sued for anything they’ve previously done.
I think you’re talking past each other. I interpret Nathan as saying that he could prove that everyone on earth has been harmed, but that he couldn’t do that in a safe manner.
Thanks Quintin. That’s useful. I think the general standard of holding organizations liable for any action which they do not prove to be safe is indeed a terrible idea. I do think that certain actions may carry higher implicit harms, and should be held to a higher standard of caution.
Perhaps you, or others, will give me your opinions on the following list of actions. Where is a good point, in your opinion, to ‘draw the line’? Starting from what I would consider ‘highly dangerous and much worse than Llama2’ and going down to ‘less dangerous than Llama2’, here are some related actions.
Releasing for free a software product onto the internet explicitly engineered to help create bio-weapons. Advertising this product as containing not only necessary gene sequences and lab protocols, but also explicit instructions for avoiding government and organization safety screens. Advertising that this product shows multiple ways to create your desired bio-weapon, including using your own lab equipment or deceiving Contract Research Organizations into unwittingly assisting you.
Releasing the same software product with the same information, but not mentioning to anyone what it is intended for. Because it is a software product, rather than an ML model, the information is all correct and not mixed in with hallucinations. The user only needs to navigate to the appropriate part of the app to get the information, rather than querying a language model.
Releasing a software product not actively intended for the above purposes, but which does happen contain all that information and can incidentally be used for those purposes.
Releasing an LLM which was trained on a dataset containing this information, and can regurgitate the information accurately. Furthermore, tuning this LLM before release to be happy to help users with requests to carry out a bio-weapons project, and double-checking to make sure that the information given is accurate.
Releasing an LLM that happens to contain such information and could be used for such purposes, but the information is not readily available (e.g. requires jail breaking or fine tuning) and is contaminated with hallucinations which can make it difficult to know which portion of the information to trust.
Releasing an LLM that could be used for such purposes, but that doesn’t yet contain the information. Bad actors would have to acquire the relevant information and fine-tune the model on it in order for the model to be able to effectively assist them in making a weapon of mass destruction.
Where should liability begin?
All of the below is my current personal opinion, and not intended to be normative.
Once you remove the “Advertising” part from (1), you’re left with “Releasing for free a software product onto the internet explicitly engineered to help create bio-weapons”, which means that (2) seems definitely worthy of criminal liability.
(3) turns that into “Releasing for free a software product onto the internet which contains information that may help create bio-weapons”, and is probably about where my fuzzy line for liability comes in. Some cases of (3) should be liable, some should not. If the information in it is just swept up as part of a big pile of publically available information, then no. If it contains substantial portions of non-public information about bio-weapon development, even as part of a much larger corpus of information not normally available to the public, then probably yes.
However: (4) re-introduces intent and should be criminally liable, since in this scenario it seems that the developer will happily fine-tune specifically for compliance and accuracy of bio-weapon development information. If they just fine-tune for compliance with user requests in general and do some random sampling of accuracy of information across a broad range of requests that don’t specifically include bio-weapons development, we’re back to the (3) case where they should be liable in some cases but not others.
(5) and (6) seem no worse than Google and Wikipedia and should basically never be liable.
Also given the rest of the replies, I think he means that it would be challenging for a plaintiff to safely prove that Llama 2 enables terrorists to make bioweapons, not that the alleged harm is “making open-source AI without proving it safe” or such.
Thanks Daniel, yes. To be more clear: I have evidence that I do not feel comfortable presenting which I believe would be more convincing than the evidence I do feel comfortable presenting. I am working on finding more convincing evidence which is still safe to present. I am seeking to learn what critics would consider to be cruxy evidence which might lie in the ‘safe for public discussion’ zone.
FWIW the heavy negative karma means that his answer is hidden by default, so that readers can’t easily see why OP thinks it might make sense to bring a lawsuit against Meta, which seems bad.