The cleanest argument that current-day AI models will not cause a catastrophe is probably that they lack the capability to do so. However, as capabilities improve, we’ll need new tools for ensuring that AI models won’t cause a catastrophe even if we can’t rule out the capability. Anthropic’s Responsible Scaling Policy (RSP) categorizes levels of risk of AI systems into different AI Safety Levels (ASL), and each level has associated commitments aimed at mitigating the risks. Some of these commitments take the form of affirmative safety cases, which are structured arguments that the system is safe to deploy in a given environment. Unfortunately, it is not yet obvious how to make a safety case to rule out certain threats that arise once AIs have sophisticated strategic abilities. The goal of this post is to present some candidates for what such a safety case might look like.
This is a post by Roger Grosse on Anthropic’s new Alignment Science Blog. The post is full of disclaimers about how it isn’t an official plan and doesn’t speak for the org (and that it’s inadequate: “none of the sketches presented here fully succeeds in addressing the sabotage risk”). But presumably it represents Anthropic’s thinking and is their working/best sketch of ASL-4 safety cases.
The three safety cases are Mechanistic Interpretability, AI Control, and Incentives Analysis.
Regardless of how good these safety cases are, it’s good when labs share their thinking on safety stuff; yay Anthropic.
Anthropic: Three Sketches of ASL-4 Safety Case Components
Link post
This is a post by Roger Grosse on Anthropic’s new Alignment Science Blog. The post is full of disclaimers about how it isn’t an official plan and doesn’t speak for the org (and that it’s inadequate: “none of the sketches presented here fully succeeds in addressing the sabotage risk”). But presumably it represents Anthropic’s thinking and is their working/best sketch of ASL-4 safety cases.
The three safety cases are Mechanistic Interpretability, AI Control, and Incentives Analysis.
Regardless of how good these safety cases are, it’s good when labs share their thinking on safety stuff; yay Anthropic.