@ryan_greenblatt one thing I’m curious about is when/how the government plays a role in your plan.
I think Sam is likely correct in pointing out that the influence exerted by you (as an individual), Sam (as an individual), or even Anthropic (as an institution) likely goes down considerably if/once governments get super involved.
I still agree with your point about how having an exit plan is still valuable (and indeed I do expect governments to be asking technical experts about their opinions RE what to do, though I also expect a bunch of DC people who know comparatively little about frontier AI systems but have long-standing relationships in the national security world will have a lot of influence.)
My guess is that you think heavy government involvement should occur for before/during the creation of ASL-4 systems, since you’re pretty concerned about risks from ASL-4 systems being developed in non-SL5 contexts.
In general, I’d be interested in seeing more about how you (and Buck) are thinking about policy stuff + government involvement. My impression is that you two have spent a lot of time thinking about how AI control fits into a broader strategic context, with that broader strategic context depending a lot on how governments act/react.
And I suspect readers will be better able to evaluate the AI control plan if some of the assumptions/expectations around government involvement are spelled out more clearly. (Put differently, I think it’s pretty hard to evaluate “how excited should I be about the AI control agenda” without understanding who is responsible for doing the AI control stuff, what’s going on with race dynamics, etc.)
My guess is that you think heavy government involvement should occur for before/during the creation of ASL-4 systems, since you’re pretty concerned about risks from ASL-4 systems being developed in non-SL5 contexts.
Yes, I think heavy government should occur once AIs can substantially accelerate general purpose R&D and AI R&D in particular. I think the occurs at some point during ASL-4.
In practice, there might be a lag between when government should get involve and when it really does get involved such that I think companies should be prepared to implement SL5 without heavy government assistance. I think SL5 with involve massive operating cost, particularly if implemented on short notice, but should be possible for a competent actor to implement with a big effort.
(I’m also somewhat skeptical that the government will actually be that helpful in implementing SL5 relative to just hiring people the relevant expertise who will often be formerly working for various government. The difficulty in SL5 implementation also depends heavily on what costs you’re willing to accept: full airgapping is conceptually simple and should be workable, but prevents serving a public API.)
In general, I’d be interested in seeing more about how you (and Buck) are thinking about policy stuff + government involvement.
I don’t think we should get into this here, but we are in fact thinking about these topics and will likely discuss this more in future posts.
And I suspect readers will be better able to evaluate the AI control plan if some of the assumptions/expectations around government involvement are spelled out more clearly.
Agreed, though I think that “do something like control” is more robust than “the AI control plan” (which we haven’t even really clearly spelled out publicly, though we do have something in mind).
@ryan_greenblatt one thing I’m curious about is when/how the government plays a role in your plan.
I think Sam is likely correct in pointing out that the influence exerted by you (as an individual), Sam (as an individual), or even Anthropic (as an institution) likely goes down considerably if/once governments get super involved.
I still agree with your point about how having an exit plan is still valuable (and indeed I do expect governments to be asking technical experts about their opinions RE what to do, though I also expect a bunch of DC people who know comparatively little about frontier AI systems but have long-standing relationships in the national security world will have a lot of influence.)
My guess is that you think heavy government involvement should occur for before/during the creation of ASL-4 systems, since you’re pretty concerned about risks from ASL-4 systems being developed in non-SL5 contexts.
In general, I’d be interested in seeing more about how you (and Buck) are thinking about policy stuff + government involvement. My impression is that you two have spent a lot of time thinking about how AI control fits into a broader strategic context, with that broader strategic context depending a lot on how governments act/react.
And I suspect readers will be better able to evaluate the AI control plan if some of the assumptions/expectations around government involvement are spelled out more clearly. (Put differently, I think it’s pretty hard to evaluate “how excited should I be about the AI control agenda” without understanding who is responsible for doing the AI control stuff, what’s going on with race dynamics, etc.)
Yes, I think heavy government should occur once AIs can substantially accelerate general purpose R&D and AI R&D in particular. I think the occurs at some point during ASL-4.
In practice, there might be a lag between when government should get involve and when it really does get involved such that I think companies should be prepared to implement SL5 without heavy government assistance. I think SL5 with involve massive operating cost, particularly if implemented on short notice, but should be possible for a competent actor to implement with a big effort.
(I’m also somewhat skeptical that the government will actually be that helpful in implementing SL5 relative to just hiring people the relevant expertise who will often be formerly working for various government. The difficulty in SL5 implementation also depends heavily on what costs you’re willing to accept: full airgapping is conceptually simple and should be workable, but prevents serving a public API.)
I don’t think we should get into this here, but we are in fact thinking about these topics and will likely discuss this more in future posts.
Agreed, though I think that “do something like control” is more robust than “the AI control plan” (which we haven’t even really clearly spelled out publicly, though we do have something in mind).