This led to people warnings, including from people like Tyler Cowen and Dean Ball, that if you create regulators, they will go looking for things to regulate, and inevitably do lots of other things over time.
I don’t see how the new form of the bill prevents the state of CA from using the law to regulate things that have nothing to do with existential risk.
Suppose you are Gavin Newsom and one day while scrolling X you see this image.
“This is misinformation! This is a threat to democracy!” you scream.
So you call up the attorney general on the phone and say “We’ve got to shut down Grok!” (pretend for the sake of argument Grok3 is a covered model, which it almost certainly will be)
The attorney general calmly reminds Newsom that “SB-1047 is only deals with catastrophic harm, not mean posts on X.”
“What could be more catastrophic!” Newsom rages. “The future of the nation is at stake!”
The bill is very specific on what catastrophic* means (creating weapons of mass destruction, $500 million in damages, or mass casualties). Maybe a court can argue this causes $500 million in damages, but, like, that seems like a real stretch to me. The attorney general could argue it in court but I don’t think they’d win.
*well, it actually defines “critical harm”, not “catastrophic”. But, you get the idea.
your faith in the likelihood that the supreme court of California will interpret this “as intended by Zvi” and not “in whatever way seems politically convenient” is much higher than mine.
I am rather confident that the California Supreme Court (or US Supreme Court, potentially) would rule that the law says what it says, and would happily bet on that.
If you think we simply don’t have any law and people can do what they want, when nothing matters. Indeed, I’d say it would be more likely to work for Gavin to today simply declare some sort of emergency about this, than to try and invoke SB 1047.
My claim is not that the supreme court will literally ignore the text of the law, but rather that phrases like “Other grave harms to public safety and security” could easily be interpreted to cover the above scenario.
If this were a federal law, I would at least have some solace that the natural checks-and-balances might take effect. But given that single-party-control of CA is unlikely to end anytime soon, giving a state law a veto over all frontier models in the United States seems bad.
CA does not have a particularly good track-record of respecting my rights. I would have the same objection if TX tried to pass a law asserting nationwide control over an industry.
I suspect this law would eventually get struck down by the US Supreme Court as a violation of interstate commerce if they actually tried to enforce it against a company that did not have employees in their state, but in the meantime the chilling effect on speech/technology would be significant.
As far as a bet, because I expect most of the effect to happen through “chilling effect” or “guidance”, it would have to be something along the lines of: the FMB will issue guidance about “best practices” that will include topics such as “misinformation” “deceptive imagery” or other topics that encourage models to censor their outputs on topics not clearly related to CRBN or Hacking.
“if SB 1047 is passed in its current from and not struck down by the Supreme Court or otherwise modified the FMB will issue guidance about “best practices” that will include topics such as “misinformation” “deceptive imagery” or other topics that encourage models to censor their outputs on topics not clearly related to CRBN or Hacking (prior to AGI, assuming it happens >3 years from now).”
Worth noticing that is a much weaker claim. The FMB issuing non-binding guidance on X is not the same as a judge holding a company liable for ~X under the law.
Mind adding an addendum to your article along the lines of “it can be reasonably speculated that the FMB will a chilling effect on freedom of speech by issuing guidance about model outputs”?
I don’t see how the new form of the bill prevents the state of CA from using the law to regulate things that have nothing to do with existential risk.
Suppose you are Gavin Newsom and one day while scrolling X you see this image.
“This is misinformation! This is a threat to democracy!” you scream.
So you call up the attorney general on the phone and say “We’ve got to shut down Grok!” (pretend for the sake of argument Grok3 is a covered model, which it almost certainly will be)
The attorney general calmly reminds Newsom that “SB-1047 is only deals with catastrophic harm, not mean posts on X.”
“What could be more catastrophic!” Newsom rages. “The future of the nation is at stake!”
How exactly do you think that conversation ends?
The bill is very specific on what catastrophic* means (creating weapons of mass destruction, $500 million in damages, or mass casualties). Maybe a court can argue this causes $500 million in damages, but, like, that seems like a real stretch to me. The attorney general could argue it in court but I don’t think they’d win.
*well, it actually defines “critical harm”, not “catastrophic”. But, you get the idea.
your faith in the likelihood that the supreme court of California will interpret this “as intended by Zvi” and not “in whatever way seems politically convenient” is much higher than mine.
I am rather confident that the California Supreme Court (or US Supreme Court, potentially) would rule that the law says what it says, and would happily bet on that.
If you think we simply don’t have any law and people can do what they want, when nothing matters. Indeed, I’d say it would be more likely to work for Gavin to today simply declare some sort of emergency about this, than to try and invoke SB 1047.
My claim is not that the supreme court will literally ignore the text of the law, but rather that phrases like “Other grave harms to public safety and security” could easily be interpreted to cover the above scenario.
If this were a federal law, I would at least have some solace that the natural checks-and-balances might take effect. But given that single-party-control of CA is unlikely to end anytime soon, giving a state law a veto over all frontier models in the United States seems bad.
CA does not have a particularly good track-record of respecting my rights. I would have the same objection if TX tried to pass a law asserting nationwide control over an industry.
I suspect this law would eventually get struck down by the US Supreme Court as a violation of interstate commerce if they actually tried to enforce it against a company that did not have employees in their state, but in the meantime the chilling effect on speech/technology would be significant.
As far as a bet, because I expect most of the effect to happen through “chilling effect” or “guidance”, it would have to be something along the lines of: the FMB will issue guidance about “best practices” that will include topics such as “misinformation” “deceptive imagery” or other topics that encourage models to censor their outputs on topics not clearly related to CRBN or Hacking.
Sounds like a good opportunity for a concrete bet, and/or manifold market?
“if SB 1047 is passed in its current from and not struck down by the Supreme Court or otherwise modified the FMB will issue guidance about “best practices” that will include topics such as “misinformation” “deceptive imagery” or other topics that encourage models to censor their outputs on topics not clearly related to CRBN or Hacking (prior to AGI, assuming it happens >3 years from now).”
@zvi ?
Worth noticing that is a much weaker claim. The FMB issuing non-binding guidance on X is not the same as a judge holding a company liable for ~X under the law.
Worth noticing that you aren’t taking the bet.
Mind adding an addendum to your article along the lines of “it can be reasonably speculated that the FMB will a chilling effect on freedom of speech by issuing guidance about model outputs”?