@AnthonyC I may be mistaken, but I took @M. Y. Zuo to be offering a reductio ad absurdum response to your comment about not being indifferent between the two ways of dying. The ‘which is a worse way to die’ debate doesn’t respond to what I wrote. I said
With respect to the survival prospects for the average human, this [whether or not the dying occurs by AGI] seems to me to be a minor detail.
I did not say that no one should care about the difference.
But the two risks are not in competition, they are complementary. If your concern about misalignment is based on caring about the continuation of the human species, and you don’t actually care how many humans other humans would kill in a successful alignment(-as-defined-here) scenario, a credible humans-kill-most-humans risk is still really helpful to your cause, because you can ally yourself with the many rational humans who don’t want to be killed either way to prevent both outcomes by killing AI in its cradle.
@AnthonyC I may be mistaken, but I took @M. Y. Zuo to be offering a reductio ad absurdum response to your comment about not being indifferent between the two ways of dying. The ‘which is a worse way to die’ debate doesn’t respond to what I wrote. I said
I did not say that no one should care about the difference.
But the two risks are not in competition, they are complementary. If your concern about misalignment is based on caring about the continuation of the human species, and you don’t actually care how many humans other humans would kill in a successful alignment(-as-defined-here) scenario, a credible humans-kill-most-humans risk is still really helpful to your cause, because you can ally yourself with the many rational humans who don’t want to be killed either way to prevent both outcomes by killing AI in its cradle.