Matthew I think you’re missing a pretty important consideration here, which is that all of these policy/governance actions did not “just happen”—a huge amount of effort has been put into them, much of it by the extended AI safety community, without which I think we would be in a very different situation. So I take almost the opposite lesson from what’s happening: concerted effort to actually try to govern AI might actually succeed—but we should be doubling down on what we are doing right and learning from what we are doing wrong, not being complacent.
To test this claim we could look to China, where AI x-risk concerns are less popular and influential. China passed a regulation on deepfakes in January 2022 and one on recommendation algorithms in March 2022. This year, they passed a regulation on generative AI which requires evaluation of training data and red teaming of model outputs. Perhaps this final measure was the result of listening to ARC and other AI safety folks who popularized model evaluations, but more likely, it seems that red teaming and evaluations are the common sense way for a government to prevent AI misbehavior.
The European Union’s AI Act was also created before any widespread recognition of AI x-risks.
On the other hand, I agree that key provisions in Biden’s executive order appear acutely influenced by AI x-risk concerns. I think it’s likely that without influence from people concerned about x-risk, their actions would more closely resemble the Blueprint for an AI Bill of Rights.
The lesson I draw is that there is plenty of appetite for AI regulation independent of x-risk concerns. But it’s important to make sure that regulation is effective, rather than blunt and untargeted.
The lesson I draw is that there is plenty of appetite for AI regulation independent of x-risk concerns. But it’s important to make sure that regulation is effective, rather than blunt and untargeted.
Yes, this is the lesson I draw too, and it’s precisely what I argue for in the post.
“CESI’s Artificial Intelligence Standardization White Paper released in 2018 states that “AI systems that have a direct impact on the safety of humanity and the safety of life, and may constitute threats to humans” must be regulated and assessed, suggesting a broad threat perception (Section 4.5.7).42 In addition, a TC260 white paper released in 2019 on AI safety/security worries that “emergence” (涌现性) by AI algorithms can exacerbate the black box effect and “autonomy” can lead to algorithmic “self-improvement” (Section 3.2.1.3).43” From https://concordia-consulting.com/wp-content/uploads/2023/10/State-of-AI-Safety-in-China.pdf
Strong agree to this, and I’d go further — I think most of the wins from the last few months wouldn’t have happened if not for the efforts of people in the AI safety ecosystem; and as lobbying efforts from those opposed to regulation heat up, we’ll need even more people advocating for this stuff.
I agree we should not be complacent. I think there’s a difference between being complacent and moving our focus to problems that are least likely to be solved by default. My primary message here is that we should re-evaluate which problems need concerted effort now, and potentially move resources to different parts of the problem—or different problems entirely—after we have reassessed. I am asking people to raise the bar for what counts as “concerted effort to actually try to govern AI”, which I think pushes against some types of blanket advocacy that merely raise awareness, and some proposals that (in my opinion) lack nuance.
Any chance that you could make this more concrete by specifying such a proposal? I expect it’d be possible to make up an example if you want to avoid criticising any specific project.
I have seen several people say that EAs should focus on promoting stupid legislation that slows down AI incidentally, since that’s “our best hope” to make sure things go well. In one of the footnotes, I cited an example of someone making this argument.
While this example could be dismissed as a weakman, I’ve also seen more serious proposals that I believe share both this theme and tone. This is how I currently perceive some of the “AI pause” proposals, especially those that fail to specify a mechanism to adjust regulatory strictness in response to new evidence. Nonetheless, I acknowledge that my disagreement with these proposals often comes down to a more fundamental disagreement about the difficulty of alignment, rather than any beliefs about the social response to AI risk.
Matthew I think you’re missing a pretty important consideration here, which is that all of these policy/governance actions did not “just happen”—a huge amount of effort has been put into them, much of it by the extended AI safety community, without which I think we would be in a very different situation. So I take almost the opposite lesson from what’s happening: concerted effort to actually try to govern AI might actually succeed—but we should be doubling down on what we are doing right and learning from what we are doing wrong, not being complacent.
To test this claim we could look to China, where AI x-risk concerns are less popular and influential. China passed a regulation on deepfakes in January 2022 and one on recommendation algorithms in March 2022. This year, they passed a regulation on generative AI which requires evaluation of training data and red teaming of model outputs. Perhaps this final measure was the result of listening to ARC and other AI safety folks who popularized model evaluations, but more likely, it seems that red teaming and evaluations are the common sense way for a government to prevent AI misbehavior.
The European Union’s AI Act was also created before any widespread recognition of AI x-risks.
On the other hand, I agree that key provisions in Biden’s executive order appear acutely influenced by AI x-risk concerns. I think it’s likely that without influence from people concerned about x-risk, their actions would more closely resemble the Blueprint for an AI Bill of Rights.
The lesson I draw is that there is plenty of appetite for AI regulation independent of x-risk concerns. But it’s important to make sure that regulation is effective, rather than blunt and untargeted.
Link to China’s red teaming standard — note that their definitions of misbehavior are quite different from yours, and they do not focus on catastrophic risks: https://twitter.com/mattsheehan88/status/1714001598383317459?s=46
Yes, this is the lesson I draw too, and it’s precisely what I argue for in the post.
Full credit to you for seeing this ahead of time, I’ve been surprised by the appetite for regulation.
“CESI’s Artificial Intelligence Standardization White Paper released in 2018 states
that “AI systems that have a direct impact on the safety of humanity and the safety of life,
and may constitute threats to humans” must be regulated and assessed, suggesting a broad
threat perception (Section 4.5.7).42 In addition, a TC260 white paper released in 2019 on AI
safety/security worries that “emergence” (涌现性) by AI algorithms can exacerbate the
black box effect and “autonomy” can lead to algorithmic “self-improvement” (Section
3.2.1.3).43”
From https://concordia-consulting.com/wp-content/uploads/2023/10/State-of-AI-Safety-in-China.pdf
Strong agree to this, and I’d go further — I think most of the wins from the last few months wouldn’t have happened if not for the efforts of people in the AI safety ecosystem; and as lobbying efforts from those opposed to regulation heat up, we’ll need even more people advocating for this stuff.
I agree we should not be complacent. I think there’s a difference between being complacent and moving our focus to problems that are least likely to be solved by default. My primary message here is that we should re-evaluate which problems need concerted effort now, and potentially move resources to different parts of the problem—or different problems entirely—after we have reassessed. I am asking people to raise the bar for what counts as “concerted effort to actually try to govern AI”, which I think pushes against some types of blanket advocacy that merely raise awareness, and some proposals that (in my opinion) lack nuance.
Any chance that you could make this more concrete by specifying such a proposal? I expect it’d be possible to make up an example if you want to avoid criticising any specific project.
I have seen several people say that EAs should focus on promoting stupid legislation that slows down AI incidentally, since that’s “our best hope” to make sure things go well. In one of the footnotes, I cited an example of someone making this argument.
While this example could be dismissed as a weakman, I’ve also seen more serious proposals that I believe share both this theme and tone. This is how I currently perceive some of the “AI pause” proposals, especially those that fail to specify a mechanism to adjust regulatory strictness in response to new evidence. Nonetheless, I acknowledge that my disagreement with these proposals often comes down to a more fundamental disagreement about the difficulty of alignment, rather than any beliefs about the social response to AI risk.
Right now it seems to me that one of the highest impact things not likely to be done by default is substantially increased funding for AI safety.
And another interesting one from the summit:
“There was almost no discussion around agents—all gen AI & model scaling concerns.
It’s perhaps because agent capabilities are mediocre today and thus hard to imagine, similar to how regulators couldn’t imagine GPT-3’s implications until ChatGPT.”—https://x.com/kanjun/status/1720502618169208994?s=46&t=D5sNUZS8uOg4FTcneuxVIg