Somewhat meta: would it not be preferable if more people accepted humanity and human values mortality/transient nature and more attention was directed towards managing the transition to whatever could be next instead of futile attempts to prevent anything that doesn’t align with human values from ever existing in this particular light cone? Is Eliezer’s strong attachment to human values a potential giant blindspot?
instead of futile attempts to prevent anything that doesn’t align with human values from ever existing in this particular light cone?
I don’t think this is futile, just very hard. In general, I think people rush far too quickly from ‘this is hard’ to ‘this is impossible’ (even in cases that look far less hard than AGI alignment).
Is Eliezer’s strong attachment to human values a potential giant blindspot?
Past-Eliezer (as of the 1990s) if anything erred in the opposite direction; I think EY’s natural impulse is toward moral cosmopolitanism rather than human parochialism or conservatism. But unrestricted paperclip maximization is bad from a cosmopolitan perspective, not just from a narrowly human or bioconservative perspective.
Somewhat meta: would it not be preferable if more people accepted humanity and human values mortality/transient nature and more attention was directed towards managing the transition to whatever could be next instead of futile attempts to prevent anything that doesn’t align with human values from ever existing in this particular light cone? Is Eliezer’s strong attachment to human values a potential giant blindspot?
I don’t think this is futile, just very hard. In general, I think people rush far too quickly from ‘this is hard’ to ‘this is impossible’ (even in cases that look far less hard than AGI alignment).
Past-Eliezer (as of the 1990s) if anything erred in the opposite direction; I think EY’s natural impulse is toward moral cosmopolitanism rather than human parochialism or conservatism. But unrestricted paperclip maximization is bad from a cosmopolitan perspective, not just from a narrowly human or bioconservative perspective.
I do see this as a blind spot, and perhaps may be giving this problem a harder task than what needs to happen.