I’m having a failure of imagination here, I assume an aligned AGI would upload human minds, preventing them from dying, but beyond that it’s not clear to me what would happen. Would we just get wireheaded? Is there a CEV from human values that results in choice or life that has an analogue to a good life right now? Does anyone understand what would happen if you gave an AGI human values and told it to extrapolate from there?
[Question] What would the creation of aligned AGI look like for us?
I’m having a failure of imagination here, I assume an aligned AGI would upload human minds, preventing them from dying, but beyond that it’s not clear to me what would happen. Would we just get wireheaded? Is there a CEV from human values that results in choice or life that has an analogue to a good life right now? Does anyone understand what would happen if you gave an AGI human values and told it to extrapolate from there?