Give more access to orgs like Redwood, Apollo, and METR (I don’t know how much access you currently give, but I suspect the globally-optimal thing would be to give more)
I agree, and I also think that this would be better implemented by government AI Safety Institutions.
Specifically, I think that AISIs should build (and make mandatory the use of) special SCIF-style reading rooms where external evaluators would be given early access to new models. This would mean that the evaluators would need permission from the government, rather than permission from AI companies. I think it’s a mistake to rely on the AI companies voluntarily giving early access to external evaluators.
I think that Anthropic could make this a lot more likely to happen if they pushed for it, and that then it wouldn’t be so hard to pull other major AI companies into the plan.
In regards to:
I agree, and I also think that this would be better implemented by government AI Safety Institutions.
Specifically, I think that AISIs should build (and make mandatory the use of) special SCIF-style reading rooms where external evaluators would be given early access to new models. This would mean that the evaluators would need permission from the government, rather than permission from AI companies. I think it’s a mistake to rely on the AI companies voluntarily giving early access to external evaluators.
I think that Anthropic could make this a lot more likely to happen if they pushed for it, and that then it wouldn’t be so hard to pull other major AI companies into the plan.