I am most interested in understanding their ASL-3 security commitments in more detail. My sense was that it was unlikely for Anthropic to stick with their original commitments there, and am curious whether they have indeed changed.
We will publish a more comprehensive list of our implemented ASL-3 security measures below (with additional components not listed here) following the [RAND] report’s publication.
The RAND report was published in late May but this list never appeared.
Also the old RSP was about “containment” rather than just “security”: containment is supposed to address risk of model self-exfiltration in addition to risk of weights being stolen. (But not really at ASL-3.) The new RSP is just about security.
Old ASL-3 security:
At ASL-3, labs should harden security against non-state attackers and provide some defense against state-level attackers. We commit to the following security themes. Similarly to ASL-2, this summary previews the key security measures at a high level and is based on the forthcoming RAND report. We will publish a more comprehensive list of our implemented ASL-3 security measures below (with additional components not listed here) following the report’s publication.
These requirements are cumulative above the ASL-2 requirements.
At the software level, there should be strict inventory management tracking all software components used in development and deployment. Adhering to specifications like SSDF and SLSA, which includes a secure build pipeline and cryptographic signature enforcement at deployment time, must provide tamper-proof infrastructure. Frequent software updates and compliance monitoring must maintain security over time.
On the hardware side, sourcing should focus on security-minded manufacturers and supply chains. Storage of sensitive weights must be centralized and restricted. Cloud network infrastructure must follow secure design patterns.
Physical security should involve sweeping premises for intrusions. Hardware should be hardened to prevent external attacks on servers and devices.
Segmentation should be implemented throughout the organization to a high threshold limiting blast radius from attacks. Access to weights should be indirect, via managed interfaces rather than direct downloads. Software should place limitations like restricting third-party services from accessing weights directly. Employees must be made aware that weight interactions are monitored. These controls should scale as an organization scales.
Ongoing monitoring such as compromise assessments and blocking of malicious queries should be both automated and manual. Limits must be placed on the number of inferences for each set of credentials. Model interactions that could bypass monitoring must be avoided.
Organizational policies must aim to enforce security through code, limiting reliance on manual compliance.
To scale to meet the risk from people-vectors, insider threat programs should be hardened to require multi-party controls and incentivize reporting risks. Endpoints should be hardened to run only allowed software.
Pen-testing, diverse security experience, concrete incident experience, and funding for substantial capacity all should contribute. A dedicated, resourced security red team with ongoing access to design and code must support testing for insider threats. Effective honeypots should be set up to detect attacks.
New ASL-3 security:
When a model must meet the ASL-3 Security Standard, we will evaluate whether the measures we have implemented make us highly protected against most attackers’ attempts at stealing model weights.
We consider the following groups in scope: hacktivists, criminal hacker groups, organized cybercrime groups, terrorist organizations, corporate espionage teams, internal employees,[1] and state-sponsored programs that use broad-based and non-targeted techniques (i.e., not novel attack chains).
The following groups are out of scope for the ASL-3 Security Standard because further testing (as discussed below) should confirm that the model would not meaningfully increase their ability to do harm: state-sponsored programs that specifically target us (e.g., through novel attack chains or insider compromise) and a small number (~10) of non-state actors with state-level resourcing or backing that are capable of developing novel attack chains that utilize 0-day attacks.
To make the required showing, we will need to satisfy the following criteria:
Threat modeling: Follow risk governance best practices, such as use of the MITRE ATT&CK Framework to establish the relationship between the identified threats, sensitive assets, attack vectors and, in doing so, sufficiently capture the resulting risks that must be addressed to protect model weights from theft attempts. As part of this requirement, we should specify our plans for revising the resulting threat model over time.
Security frameworks: Align to and, as needed, extend industry-standard security frameworks for addressing identified risks, such as disclosure of sensitive information, tampering with accounts and assets, and unauthorized elevation of privileges with the appropriate controls. This includes:
Perimeters and access controls: Building strong perimeters and access controls around sensitive assets to ensure AI models and critical systems are protected from unauthorized access. We expect this will include a combination of physical security, encryption, cloud security, infrastructure policy, access management, and weight access minimization and monitoring.
Lifecycle security: Securing links in the chain of systems and software used to develop models, to prevent compromised components from being introduced and to ensure only trusted code and hardware is used. We expect this will include a combination of software inventory, supply chain security, artifact integrity, binary authorization, hardware procurement, and secure research development lifecycle.
Monitoring: Proactively identifying and mitigating threats through ongoing and effective monitoring, testing for vulnerabilities, and laying traps for potential attackers. We expect this will include a combination of endpoint patching, product security testing, log management, asset monitoring, and intruder deception techniques.
Resourcing: Investing sufficient resources in security. We expect meeting this standard of security to require roughly 5-10% of employees being dedicated to security and security-adjacent work.
Audits: Develop plans to (1) audit and assess the design and implementation of the security program and (2) share these findings (and updates on any remediation efforts) with management on an appropriate cadence. We expect this to include independent validation of threat modeling and risk assessment results; a sampling-based audit of the operating effectiveness of the defined controls; periodic, broadly scoped, and independent testing with expert red-teamers who are industry-renowned and have been recognized in competitive challenges.
Third-party environments: Document how all relevant models will meet the criteria above, even if they are deployed in a third-party partner’s environment that may have a different set of safeguards.
The new standard is more vague and meta, and Anthropic indeed abandoned various specifics from the original RSP. I think it is very far from passing the LeCun test; it’s more like talking about security themes than making an object-level if-then commitment. I don’t think Anthropic’s security at ASL-3 is a huge deal, and I expect Anthropic be quite non-LeCun-y, but I think this is just too vague for me to feel good about.
In May Anthropic said “around 8% of all Anthropic employees are now working on security-adjacent areas and we expect that proportion to grow further as models become more economically valuable to attackers.” Apparently they don’t expect to increase that for ASL-3.
Edit: I changed my mind. I agree with Ryan’s comment below. I wish we lived in a world where it’s optimal to make strong object-level if-then commitments, but we don’t, since we don’t know which mitigations will be best to focus on. Tying hands to implement specific mitigations would waste resources. Better to make more meta commitments. (Strong versions require external auditing.)
We will implement robust insider risk controls to mitigate most insider risk, but consider mitigating risks from highly sophisticated state-compromised insiders to be out of scope for ASL-3. We are committed to further enhancing these protections as a part of our ASL-4 preparations.
I agree on abandoning various specifics, but I would note that the new standard is much more specific (less vague) on what needs to be defended against and what the validation process and threat modeling process should be.
(E.g., rather than “non-state actors”, the RSP more specifically says which groups are and aren’t in scope.)
I overall think the new proposal is notably less vague on the most important aspects, though I agree it won’t pass the LeCun test due to insufficiently precise guidance around auditing. Hopefully this can be improved with future version or for future ASLs.
Oops, I forgot https://www.anthropic.com/rsp-updates. This is great. I really like that Anthropic shares “non-binding descriptions of our future ASL-3 safeguard plans.”
I am most interested in understanding their ASL-3 security commitments in more detail. My sense was that it was unlikely for Anthropic to stick with their original commitments there, and am curious whether they have indeed changed.
Old ASL-3 standard: see original RSP pp. 7-8 and 21-22. New ASL-3 standard. Quotes below.
Also note the original RSP said
The RAND report was published in late May but this list never appeared.
Also the old RSP was about “containment” rather than just “security”: containment is supposed to address risk of model self-exfiltration in addition to risk of weights being stolen. (But not really at ASL-3.) The new RSP is just about security.
Old ASL-3 security:
New ASL-3 security:
The new standard is more vague and meta, and Anthropic indeed abandoned various specifics from the original RSP. I think it is very far from passing the LeCun test; it’s more like talking about security themes than making an object-level if-then commitment. I don’t think Anthropic’s security at ASL-3 is a huge deal, and I expect Anthropic be quite non-LeCun-y, but I think this is just too vague for me to feel good about.
In May Anthropic said “around 8% of all Anthropic employees are now working on security-adjacent areas and we expect that proportion to grow further as models become more economically valuable to attackers.” Apparently they don’t expect to increase that for ASL-3.
Edit: I changed my mind. I agree with Ryan’s comment below. I wish we lived in a world where it’s optimal to make strong object-level if-then commitments, but we don’t, since we don’t know which mitigations will be best to focus on. Tying hands to implement specific mitigations would waste resources. Better to make more meta commitments. (Strong versions require external auditing.)
We will implement robust insider risk controls to mitigate most insider risk, but consider mitigating risks from highly sophisticated state-compromised insiders to be out of scope for ASL-3. We are committed to further enhancing these protections as a part of our ASL-4 preparations.
I agree on abandoning various specifics, but I would note that the new standard is much more specific (less vague) on what needs to be defended against and what the validation process and threat modeling process should be.
(E.g., rather than “non-state actors”, the RSP more specifically says which groups are and aren’t in scope.)
I overall think the new proposal is notably less vague on the most important aspects, though I agree it won’t pass the LeCun test due to insufficiently precise guidance around auditing. Hopefully this can be improved with future version or for future ASLs.
Oops, I forgot https://www.anthropic.com/rsp-updates. This is great. I really like that Anthropic shares “non-binding descriptions of our future ASL-3 safeguard plans.”