Parenthetically, no act powerful enough and gameboard-flipping enough to qualify is inside the Overton Window of politics, or possibly even of effective altruism, which presents a separate social problem. I usually dodge around this problem by picking an exemplar act which is powerful enough to actually flip the gameboard, but not the most alignable act because it would require way too many aligned details: Build self-replicating open-air nanosystems and use them (only) to melt all GPUs.
Since any such nanosystems would have to operate in the full open world containing lots of complicated details, this would require tons and tons of alignment work, is not the pivotal act easiest to align, and we should do some other thing instead. But the other thing I have in mind is also outside the Overton Window, just like this is. So I use “melt all GPUs” to talk about the requisite power level and the Overton Window problem level, both of which seem around the right levels to me, but the actual thing I have in mind is more alignable; and this way, I can reply to anyone who says “How dare you?!” by saying “Don’t worry, I don’t actually plan on doing that.”
As an aside:
What? How if someone is offended that you’re advocating for an action that’s as norm violating and uh, to put it frankly, an act of war, as burning all the GPUs, does it really ameliorate their concerns to say “don’t worry, I’m not actually advocating that someone do that. I’m only advocating that they do something similarly extreme, which I’m telling you outright is also outside the overton window.”
Has this reassured a single concerned person, ever?
It reassures me, and I think it’s the right thing to do in this case, because policy discussions follow strong contextualizing norms. Using a layer of indirection, as Eliezer does here, makes it clearer that this is a theoretical discussion, rather than an attempt to actually advocate for that specific intervention.
As an aside:
What? How if someone is offended that you’re advocating for an action that’s as norm violating and uh, to put it frankly, an act of war, as burning all the GPUs, does it really ameliorate their concerns to say “don’t worry, I’m not actually advocating that someone do that. I’m only advocating that they do something similarly extreme, which I’m telling you outright is also outside the overton window.”
Has this reassured a single concerned person, ever?
It reassures me, and I think it’s the right thing to do in this case, because policy discussions follow strong contextualizing norms. Using a layer of indirection, as Eliezer does here, makes it clearer that this is a theoretical discussion, rather than an attempt to actually advocate for that specific intervention.