“If the world were unified around the priority of minimizing global catastrophic risk, I think that we could reduce risk significantly further by implementing a global, long-lasting, and effectively enforced pause on frontier AI development—including a moratorium on the development and production of some types of computing hardware”
This really needs to be shouted from the rooftops. In the public sphere, people will hear “responsible scaling policy” as “It’s maximally safe to keep pushing ahead with AI” rather than “We are taking on huge risks because politicians can’t be bothered to coordinate”.
This really needs to be shouted from the rooftops.
I disagree. I think it’s important that we shout from the rooftops that the existential risk from AI is real, but I disagree that we should shout from the rooftops that a sufficiently good pause would solve it (even though I agree with Paul that it is true). I talk about this in this comment.
Historically, I think that a lot of causes have been hurt by a sort of purity-testing where scientists are forced to endorse the most extreme policy, even if it’s not the best policy, on the idea that it would solve the problem in theory if you had a magic button that enacted it. Consider, for example, the idea that climate scientists should all have to endorse the idea that, if we ended capitalism, it would solve climate change. Though true, I do not think that would help the cause of climate change! Even if climate change were enough of an existential risk that it was worth sacrificing our entire economy for it (as is maybe true of AI risk), it would still not be the case that advocating for that would be at all helpful, because there are much more effective ways of addressing climate change that starting a communist revolution.
I think everyone should be clear about what they think the risks are, but I think forcing people to publicly endorse policies that they don’t endorse in practice just because they would solve the problem in theory is not a recipe for policy success.
I think forcing people to publicly endorse policies
Saying “If this happened, it would solve the problem” is not to endorse a policy. (though perhaps literally shouting from the rooftops might be less than ideal)
It’s entirely possible to state both “If x happened, it’d solve the problem”, and “The policy we think is most likely to be effective in practice is Y”. They can be put in the same statement quite simply.
It’s reasonable to say that this might not be the most effective communication strategy. (though I think on balance I’d disagree) It’s not reasonable to say that this amounts to publicly endorsing a policy.
...if we ended capitalism, it would solve climate change. Though true...
This seems an unhelpful parallel, first because it’s not clearly true. (In particular “ended capitalism” isn’t “ended capitalism, and replaced it with communism”, nor “ended capitalism overnight without a plan to replace it”). Second, because the proposal in this case is to notactively enact a radically disruptive change to society.
The logic of the point you’re making is reasonable, but the parallel has a bunch of baggage that reduces overall clarity IMO.
...because there are much more effective ways of addressing climate change than starting a communist revolution...
This part isn’t even a parallel: here even if successful the communist revolution wouldn’t be most effective. However if successful a sufficiently good pause would be most effective.
It’s entirely possible to state both “If x happened, it’d solve the problem”, and “The policy we think is most likely to be effective in practice is Y”. They can be put in the same statement quite simply.
That’s a lot of nuance that you’re trying to convey to the general public, which is a notoriously hard thing to do.
Hmm. Perhaps the thing I’d endorse is more [include this in every detailed statement about policy/regulation], rather than [shout it from the rooftops].
So, for example, if the authors agree with the statement, I think this should be in:
ARC Evals’ RSP post.
Every RSP.
Proposals for regulation.
...
I’m fine if we don’t start printing it on bumper stickers.
The outcome I’m interested in is something like: every person with significant influence on policy knows that this is believed to be a good/ideal solution, and that the only reasons against it are based on whether it’s achievable in the right form.
If ARC Evals aren’t saying this, RSPs don’t include it, and many policy proposals don’t include it..., then I don’t expect this to become common knowledge. We’re much less likely to get a stop if most people with influence don’t even realize it’s the thing that we’d ideally get.
This really needs to be shouted from the rooftops. In the public sphere, people will hear “responsible scaling policy” as “It’s maximally safe to keep pushing ahead with AI” rather than “We are taking on huge risks because politicians can’t be bothered to coordinate”.
I disagree. I think it’s important that we shout from the rooftops that the existential risk from AI is real, but I disagree that we should shout from the rooftops that a sufficiently good pause would solve it (even though I agree with Paul that it is true). I talk about this in this comment.
Historically, I think that a lot of causes have been hurt by a sort of purity-testing where scientists are forced to endorse the most extreme policy, even if it’s not the best policy, on the idea that it would solve the problem in theory if you had a magic button that enacted it. Consider, for example, the idea that climate scientists should all have to endorse the idea that, if we ended capitalism, it would solve climate change. Though true, I do not think that would help the cause of climate change! Even if climate change were enough of an existential risk that it was worth sacrificing our entire economy for it (as is maybe true of AI risk), it would still not be the case that advocating for that would be at all helpful, because there are much more effective ways of addressing climate change that starting a communist revolution.
I think everyone should be clear about what they think the risks are, but I think forcing people to publicly endorse policies that they don’t endorse in practice just because they would solve the problem in theory is not a recipe for policy success.
Saying “If this happened, it would solve the problem” is not to endorse a policy. (though perhaps literally shouting from the rooftops might be less than ideal)
It’s entirely possible to state both “If x happened, it’d solve the problem”, and “The policy we think is most likely to be effective in practice is Y”. They can be put in the same statement quite simply.
It’s reasonable to say that this might not be the most effective communication strategy. (though I think on balance I’d disagree)
It’s not reasonable to say that this amounts to publicly endorsing a policy.
This seems an unhelpful parallel, first because it’s not clearly true. (In particular “ended capitalism” isn’t “ended capitalism, and replaced it with communism”, nor “ended capitalism overnight without a plan to replace it”).
Second, because the proposal in this case is to not actively enact a radically disruptive change to society.
The logic of the point you’re making is reasonable, but the parallel has a bunch of baggage that reduces overall clarity IMO.
This part isn’t even a parallel: here even if successful the communist revolution wouldn’t be most effective. However if successful a sufficiently good pause would be most effective.
That’s a lot of nuance that you’re trying to convey to the general public, which is a notoriously hard thing to do.
Hmm. Perhaps the thing I’d endorse is more [include this in every detailed statement about policy/regulation], rather than [shout it from the rooftops].
So, for example, if the authors agree with the statement, I think this should be in:
ARC Evals’ RSP post.
Every RSP.
Proposals for regulation.
...
I’m fine if we don’t start printing it on bumper stickers.
The outcome I’m interested in is something like: every person with significant influence on policy knows that this is believed to be a good/ideal solution, and that the only reasons against it are based on whether it’s achievable in the right form.
If ARC Evals aren’t saying this, RSPs don’t include it, and many policy proposals don’t include it..., then I don’t expect this to become common knowledge.
We’re much less likely to get a stop if most people with influence don’t even realize it’s the thing that we’d ideally get.