Those are good points. There are some considerations that go in the other direction. Sometimes it’s not obvious what’s a “failure to convince people” vs. “a failure of some people to be convincible.” (I mean convincible by object-level arguments as opposed to convincible through social cascades where a particular new view reaches critical mass.)
I believe both of the following:
Persuasion efforts haven’t been exhausted yet: we can do better at reaching not-yet-safety-concerned AI researchers. (That said, I think it’s at least worth considering that we’re getting close to exhausting low-hanging fruit?)
Even so, “persuasion as the main pillar of a strategy” is somewhat likely to be massively inadequate because it’s difficult to change the minds and culture of humans in general (even if they’re smart), let alone existing organizations.
Another point that’s maybe worth highlighting is that the people who could make large demands don’t have to be the same people who are best-positioned for making smaller asks. (This is Katja’s point about there not being a need for everyone to coordinate into a single “we.”) The welfarism vs. abolitionism debate in animal advocacy and discussion of the radical flank effect seems related. I also agree with a point lc makes in his post on slowing down AI. He points out that there’s arguably a “missing mood” around the way most people in EA and the AI alignment community communicate with safety-unconcerned researchers. The missing sense of urgency probably lowers the chance of successful persuasion efforts?
Lastly, it’s a challenge that there’s little consensus in the EA research community around important questions like “How hard is AI alignment?,” “How hard is alignment conditional on <5 years to TAI?,” and “How long are TAI timelines?” (Though maybe there’s quite some agreement on the second one and the answer is at least, “it’s not easy?”)
I’d imagine there would at least be quite a strong EA expert consensus on the following conditional statement (which has both normative and empirical components):
Let’s call it “We’re in Inconvenient World” if it’s true that, absent strong countermeasures, we’ll have misaligned AI that brings about human extinction in <5 years. If the chance that we’re in Inconvenient World is10% or higher, we should urgently make large changes to the way AI development progresses as a field/industry.
Based on this, some further questions one could try to estimate are:
How many people (perhaps weighted by their social standing within an organization, opinion leaders, etc.) are convincible of the above conditional statement? Is it likely we could reach a critical mass?
Doing this for any specific org (or relevant branch of government, etc.) that seems to play a central role
What’s the minimum consensus threshold for “We’re in Inconvenient World?” (I.e., what percentage would be indefensibly low to believe in light of peer disagreement unless one considers oneself the world’s foremost authority on the question?)
He points out that there’s arguably a “missing mood” around the way most people in EA and the AI alignment community communicate with safety-unconcerned researchers. The missing sense of urgency probably lowers the chance of successful persuasion efforts?
Sorry for responding very late, but it’s basically because contra the memes, most LWers do not agree with Eliezer’s views on how doomed we are. This is very much a fringe viewpoint on LW, not the mainstream.
So the missing mood is basically because most of LW doesn’t share Eliezer’s views on certain cruxes.
Those are good points. There are some considerations that go in the other direction. Sometimes it’s not obvious what’s a “failure to convince people” vs. “a failure of some people to be convincible.” (I mean convincible by object-level arguments as opposed to convincible through social cascades where a particular new view reaches critical mass.)
I believe both of the following:
Persuasion efforts haven’t been exhausted yet: we can do better at reaching not-yet-safety-concerned AI researchers. (That said, I think it’s at least worth considering that we’re getting close to exhausting low-hanging fruit?)
Even so, “persuasion as the main pillar of a strategy” is somewhat likely to be massively inadequate because it’s difficult to change the minds and culture of humans in general (even if they’re smart), let alone existing organizations.
Another point that’s maybe worth highlighting is that the people who could make large demands don’t have to be the same people who are best-positioned for making smaller asks. (This is Katja’s point about there not being a need for everyone to coordinate into a single “we.”) The welfarism vs. abolitionism debate in animal advocacy and discussion of the radical flank effect seems related. I also agree with a point lc makes in his post on slowing down AI. He points out that there’s arguably a “missing mood” around the way most people in EA and the AI alignment community communicate with safety-unconcerned researchers. The missing sense of urgency probably lowers the chance of successful persuasion efforts?
Lastly, it’s a challenge that there’s little consensus in the EA research community around important questions like “How hard is AI alignment?,” “How hard is alignment conditional on <5 years to TAI?,” and “How long are TAI timelines?” (Though maybe there’s quite some agreement on the second one and the answer is at least, “it’s not easy?”)
I’d imagine there would at least be quite a strong EA expert consensus on the following conditional statement (which has both normative and empirical components):
Based on this, some further questions one could try to estimate are:
How many people (perhaps weighted by their social standing within an organization, opinion leaders, etc.) are convincible of the above conditional statement? Is it likely we could reach a critical mass?
Doing this for any specific org (or relevant branch of government, etc.) that seems to play a central role
What’s the minimum consensus threshold for “We’re in Inconvenient World?” (I.e., what percentage would be indefensibly low to believe in light of peer disagreement unless one considers oneself the world’s foremost authority on the question?)
Sorry for responding very late, but it’s basically because contra the memes, most LWers do not agree with Eliezer’s views on how doomed we are. This is very much a fringe viewpoint on LW, not the mainstream.
So the missing mood is basically because most of LW doesn’t share Eliezer’s views on certain cruxes.