Supposing SB1047 passes, what are the main ways in which you think it will contribute to (increasing or decreasing) existential risks?
Idea #1: Transparency through SSPs
It seems to be like the main way it could reduce AI risk is by making it more likely that governments and/or members of the public are able to notice (a) signs of dangerous capabilities or (b) problems in comapnies’ safety and security plans.
One counter to this is that companies will be able to write things that sound good but don’t actually give governments or members of the public the ability to meaningfully understand what’s going on. For example, there are ways to write “we have an AI that is able to do novel AI R&D tasks” in ways. that underplay the extent or nature of the risks.
The counter to that seems to me like “well yes, we still have to deal with the fact that companies can try to make vague/unhelp SSPs [CC the discourse about RSPs] and we have to deal with the fact that governments will need to have sufficient risk context to understand the SSPs and understand when they need to intervene. You can’t expect one bill in 2024 to solve all of the major challenges, but it pushes in the right direction and on the margin more transparency seems good.”
Idea #2: Whistleblower mechanisms
The whistleblower protections seem great– and perhaps even better than “whistleblower protections” are “whistleblower mechanisms.” I am worried about the amount of friction RE whistleblowing– who do you go to, what channels do you use, what are you supposed to say, etc etc. The fact that companies would have to provide “a reasonable internal process” and employees are explicitly told they can use this channel to disclose “misleading statements related to its safety and security protocol” or “failure to disclose known risks to employees” seems excellent.
The counter is that a whistleblower could say “I’m worried about X” and the most likely outcome is that the government ignores X because it doesn’t have enough time/attention/technical capacity/risk context to understand why it should care or what it should do. But of course the counter to that counter is once again “yes, this doesn’t fix everything, but surely we are better off in a world where clear whistleblower mechanisms exist, and hopefully there will be other efforts to make sure the government is equipped with the personnel/resources/processes it needs to notice and prepare for AI risks.”
(Meta: After the political battles around SB1047 end, I hope more people end up writing about if/how they think SB1047 will/would affect AI risks. So much of the discourse got [perhaps rightfully] hijacked by political fights about claims that seem pretty ridiculous. It’s plausible that this was the right call– political battles need to be fought sometimes– but on the margin I wish there was more “Here’s the affirmative/positive case for why SB1047 is good”, “Here are some concerns I have about SB1047 from the perspective of reducing existential risk, not from the perspective of responding to industry talking points”, and things like “SB1047 seems to help but only if governments also develop X and Y in the upcoming years, and here are some thoughts on how they could do that.”
Supposing SB1047 passes, what are the main ways in which you think it will contribute to (increasing or decreasing) existential risks?
Idea #1: Transparency through SSPs
It seems to be like the main way it could reduce AI risk is by making it more likely that governments and/or members of the public are able to notice (a) signs of dangerous capabilities or (b) problems in comapnies’ safety and security plans.
One counter to this is that companies will be able to write things that sound good but don’t actually give governments or members of the public the ability to meaningfully understand what’s going on. For example, there are ways to write “we have an AI that is able to do novel AI R&D tasks” in ways. that underplay the extent or nature of the risks.
The counter to that seems to me like “well yes, we still have to deal with the fact that companies can try to make vague/unhelp SSPs [CC the discourse about RSPs] and we have to deal with the fact that governments will need to have sufficient risk context to understand the SSPs and understand when they need to intervene. You can’t expect one bill in 2024 to solve all of the major challenges, but it pushes in the right direction and on the margin more transparency seems good.”
Idea #2: Whistleblower mechanisms
The whistleblower protections seem great– and perhaps even better than “whistleblower protections” are “whistleblower mechanisms.” I am worried about the amount of friction RE whistleblowing– who do you go to, what channels do you use, what are you supposed to say, etc etc. The fact that companies would have to provide “a reasonable internal process” and employees are explicitly told they can use this channel to disclose “misleading statements related to its safety and security protocol” or “failure to disclose known risks to employees” seems excellent.
The counter is that a whistleblower could say “I’m worried about X” and the most likely outcome is that the government ignores X because it doesn’t have enough time/attention/technical capacity/risk context to understand why it should care or what it should do. But of course the counter to that counter is once again “yes, this doesn’t fix everything, but surely we are better off in a world where clear whistleblower mechanisms exist, and hopefully there will be other efforts to make sure the government is equipped with the personnel/resources/processes it needs to notice and prepare for AI risks.”
(Meta: After the political battles around SB1047 end, I hope more people end up writing about if/how they think SB1047 will/would affect AI risks. So much of the discourse got [perhaps rightfully] hijacked by political fights about claims that seem pretty ridiculous. It’s plausible that this was the right call– political battles need to be fought sometimes– but on the margin I wish there was more “Here’s the affirmative/positive case for why SB1047 is good”, “Here are some concerns I have about SB1047 from the perspective of reducing existential risk, not from the perspective of responding to industry talking points”, and things like “SB1047 seems to help but only if governments also develop X and Y in the upcoming years, and here are some thoughts on how they could do that.”