Another concern about safety cases: they feel very vulnerable to regulatory capture. You can imagine a spectrum from “legislate that AI labs should do X, Y, Z, as enforced by regulator R” to “legislate that AI labs need to provide a safety case, which we define as anything that satisfies regulator R”. In the former case, you lose flexibility, but the remit of R is very clear. In the latter case, R has a huge amount of freedom to interpret what counts as an adequate safety case. This can backfire badly if R is not very concerned about the most important threat models; and even if they are, the flexibility makes it easier for others to apply political pressure to them (e.g. “find a reason to approve this safety case, it’s in the national interest”).
@Richard_Ngo do you have any alternative approaches in mind that are less susceptible to regulatory capture? At first glance, I think this broad argument can be applied to any situation where the government regulates anything. (There’s always some risk that R focuses on the wrong things or R experiences corporate/governmental pressure to push things through).
I do agree that the broader or more flexible the regulatory regime is, the more susceptible it might be to regulatory capture. (But again, this feels like it doesn’t really have much to do with safety cases– this is just a question of whether we want flexible or fixed/inflexible regulations in general.)
On the spectrum I outlined, the “legislate that AI labs should do X, Y, Z, as enforced by regulator R” end is less susceptible to regulatory capture (at least after the initial bill is passed).
YES, there is a tradeoff here and yes regulatory capture is real, but there are also plenty of benign agencies that balance these things fairly well. Most people on these forums live in nations where regulators do a pretty darn good job on the well-understood problems and balance these concerns fairly well. (Inside Context Problems?)
You tend to see design of regulatory processes that requires stakeholder input; in particular, the modern American regulatory state’s reliance on the Administrative Procedure Act means that it’s very difficult for a regulator to regulate without getting feedback from a wide variety of external stakeholders, ensuring that they have some flexibility without being arbitrary.
I also think, contrary to conventional wisdom, that your concern is part of why many regulators end up in a “revolving-door” mechanism—you often want individuals moving back and forth between those two worlds to cross-populate assumptions and check for areas where regulation has gotten misaligned with end goals
Another concern about safety cases: they feel very vulnerable to regulatory capture. You can imagine a spectrum from “legislate that AI labs should do X, Y, Z, as enforced by regulator R” to “legislate that AI labs need to provide a safety case, which we define as anything that satisfies regulator R”. In the former case, you lose flexibility, but the remit of R is very clear. In the latter case, R has a huge amount of freedom to interpret what counts as an adequate safety case. This can backfire badly if R is not very concerned about the most important threat models; and even if they are, the flexibility makes it easier for others to apply political pressure to them (e.g. “find a reason to approve this safety case, it’s in the national interest”).
@Richard_Ngo do you have any alternative approaches in mind that are less susceptible to regulatory capture? At first glance, I think this broad argument can be applied to any situation where the government regulates anything. (There’s always some risk that R focuses on the wrong things or R experiences corporate/governmental pressure to push things through).
I do agree that the broader or more flexible the regulatory regime is, the more susceptible it might be to regulatory capture. (But again, this feels like it doesn’t really have much to do with safety cases– this is just a question of whether we want flexible or fixed/inflexible regulations in general.)
On the spectrum I outlined, the “legislate that AI labs should do X, Y, Z, as enforced by regulator R” end is less susceptible to regulatory capture (at least after the initial bill is passed).
This is definitely a tradeoff space!
YES, there is a tradeoff here and yes regulatory capture is real, but there are also plenty of benign agencies that balance these things fairly well. Most people on these forums live in nations where regulators do a pretty darn good job on the well-understood problems and balance these concerns fairly well. (Inside Context Problems?)
You tend to see design of regulatory processes that requires stakeholder input; in particular, the modern American regulatory state’s reliance on the Administrative Procedure Act means that it’s very difficult for a regulator to regulate without getting feedback from a wide variety of external stakeholders, ensuring that they have some flexibility without being arbitrary.
I also think, contrary to conventional wisdom, that your concern is part of why many regulators end up in a “revolving-door” mechanism—you often want individuals moving back and forth between those two worlds to cross-populate assumptions and check for areas where regulation has gotten misaligned with end goals