If AGI is sufficiently nontrivial, delay of a few years might be feasible and give time to decrease doom. If AGI requires enormous datacenters, outlawing production of GPUs and treating existing ones like illegal firearms might lead to indefinite delay (though it probably takes at least an observed disaster to enter Overton window).
I see your argument moving the conclusion (pause becoming worse than no-pause) when expecting imminent AGI that only really needs modest compute and doesn’t guarantee doom. Then the only pause that works is AI-enforced one, and nobody working on that can outweigh burning some alignment progress timeline.
If AGI is sufficiently nontrivial, delay of a few years might be feasible and give time to decrease doom. If AGI requires enormous datacenters, outlawing production of GPUs and treating existing ones like illegal firearms might lead to indefinite delay (though it probably takes at least an observed disaster to enter Overton window).
I see your argument moving the conclusion (pause becoming worse than no-pause) when expecting imminent AGI that only really needs modest compute and doesn’t guarantee doom. Then the only pause that works is AI-enforced one, and nobody working on that can outweigh burning some alignment progress timeline.