The drug development comparison is an interesting one. I think a lot of policy people are operating on the comparison to nuclear, either nuclear weapons or nuclear power. Drug development is notably different because, like AI, it’s a case where the thing we want to regulate is an R&D process, not just the eventual product.
A product requirement would be “don’t publish an AI that will tell you to commit crimes.” An R&D requirement would be “At no point during training should you have an AI that’s good at telling you to commit crimes—it should always be either incompetent or safe.” But this might be hard to formalize.
It’s also hard to know how much we want more accurate measures of AI safety during a development process. We don’t really care much about GPT telling people to commit crimes—we really care about future models that model the real world and take strategic actions in it. But even if we did have a good way of measuring those capabilities during training, would we want them written into regulation? Or should we have simpler and broader restrictions on what counts as good AI development practices?
Drug development is notably different because, like AI, it’s a case where the thing we want to regulate is an R&D process, not just the eventual product
I agree, and I think I used “development” and “deployment” in this sort of vague way that didn’t highlight this very well.
But even if we did have a good way of measuring those capabilities during training, would we want them written into regulation? Or should we have simpler and broader restrictions on what counts as good AI development practices?
I think one strength of some IRB-ish models of regulation is that you don’t rely so heavily on a careful specification of the thing that’s not allowed, because instead of meshing directly with all the other bureaucratic gears, it has a layer of human judgment in between. Of course, this does pass the problem to “can you have regulatory boards that know what to look for?”, which has its own problems.
The drug development comparison is an interesting one. I think a lot of policy people are operating on the comparison to nuclear, either nuclear weapons or nuclear power. Drug development is notably different because, like AI, it’s a case where the thing we want to regulate is an R&D process, not just the eventual product.
A product requirement would be “don’t publish an AI that will tell you to commit crimes.” An R&D requirement would be “At no point during training should you have an AI that’s good at telling you to commit crimes—it should always be either incompetent or safe.” But this might be hard to formalize.
It’s also hard to know how much we want more accurate measures of AI safety during a development process. We don’t really care much about GPT telling people to commit crimes—we really care about future models that model the real world and take strategic actions in it. But even if we did have a good way of measuring those capabilities during training, would we want them written into regulation? Or should we have simpler and broader restrictions on what counts as good AI development practices?
I agree, and I think I used “development” and “deployment” in this sort of vague way that didn’t highlight this very well.
I think one strength of some IRB-ish models of regulation is that you don’t rely so heavily on a careful specification of the thing that’s not allowed, because instead of meshing directly with all the other bureaucratic gears, it has a layer of human judgment in between. Of course, this does pass the problem to “can you have regulatory boards that know what to look for?”, which has its own problems.