I think there are really two definitions of alignment here; one is “do you have close to an ideal moral system such that you would endorse it being scaled up to the whole universe for eternity” and the other is “would you commit omnicide given the opportunity.” When we talk about AI alignment, we are talking about a machine that will commit omnicide, so it doesn’t matter that much if it has the right opinions on the trolley problem or the repugnant conclusion or whatever. Hopefully no one here is planning omnicide (and probably no one on this forum would admit to desiring omnicide even if they did secretly desire it) so therefore everyone on this forum (who is not hiding a dark secret) is more aligned than Clippy, and that’s what matters right now.
I think there are really two definitions of alignment here; one is “do you have close to an ideal moral system such that you would endorse it being scaled up to the whole universe for eternity” and the other is “would you commit omnicide given the opportunity.” When we talk about AI alignment, we are talking about a machine that will commit omnicide, so it doesn’t matter that much if it has the right opinions on the trolley problem or the repugnant conclusion or whatever. Hopefully no one here is planning omnicide (and probably no one on this forum would admit to desiring omnicide even if they did secretly desire it) so therefore everyone on this forum (who is not hiding a dark secret) is more aligned than Clippy, and that’s what matters right now.