I think you have a pretty good argument against the term “accident” for misalignment risk.
Misuse risk still seems like a good description for the class of risks where—once you have AI that is aligned with its operators—those operators may try to do unsavory things with their AI, or have goals that are quite at odds with the broad values of humans and other sentient beings.
I agree somewhat, however, I think we need to be careful to distinguish “do unsavory things” from “cause human extinction”, and should generally be squarely focused on the latter. The former easily becomes too political, making coordination harder.
I think you have a pretty good argument against the term “accident” for misalignment risk.
Misuse risk still seems like a good description for the class of risks where—once you have AI that is aligned with its operators—those operators may try to do unsavory things with their AI, or have goals that are quite at odds with the broad values of humans and other sentient beings.
I agree somewhat, however, I think we need to be careful to distinguish “do unsavory things” from “cause human extinction”, and should generally be squarely focused on the latter. The former easily becomes too political, making coordination harder.