instead of futile attempts to prevent anything that doesn’t align with human values from ever existing in this particular light cone?
I don’t think this is futile, just very hard. In general, I think people rush far too quickly from ‘this is hard’ to ‘this is impossible’ (even in cases that look far less hard than AGI alignment).
Is Eliezer’s strong attachment to human values a potential giant blindspot?
Past-Eliezer (as of the 1990s) if anything erred in the opposite direction; I think EY’s natural impulse is toward moral cosmopolitanism rather than human parochialism or conservatism. But unrestricted paperclip maximization is bad from a cosmopolitan perspective, not just from a narrowly human or bioconservative perspective.
I don’t think this is futile, just very hard. In general, I think people rush far too quickly from ‘this is hard’ to ‘this is impossible’ (even in cases that look far less hard than AGI alignment).
Past-Eliezer (as of the 1990s) if anything erred in the opposite direction; I think EY’s natural impulse is toward moral cosmopolitanism rather than human parochialism or conservatism. But unrestricted paperclip maximization is bad from a cosmopolitan perspective, not just from a narrowly human or bioconservative perspective.