I understand CEV. What I don’t understand is why the programmers would ask the AI for humanity’s CEV, rather than just their own CEV.
The only (sane) reason is for signalling—it’s hard to create FAI without someone else stopping you. Given a choice, however, CEV is strictly superior. If you actually do want to have FAI then FAI will be equivalent to it. But if you just think you want FAI but it turns out that, for example, FAI gets dominated by jerks in a way you didn’t expect then FAI will end up better than FAI… even from a purely altruistic perspective.
Yeah, I’ve wondered this for a while without getting any closer to an understanding.
It seems that everything that some human “really wants” (and therefore could potentially be included in the CEV target definition) is either something that, if I was sufficiently well-informed about it, I would want for that human (in which case my CEV, properly unpacked by a superintelligence, includes it for them) or is something that, no matter how well informed I was, I would not want for that human (in which case it’s not at all clear that I ought to endorse implementing it).
If CEV-humanity makes any sense at all (which I’m not sure it does), it seems that CEV-arbitrary-subset-of-humanity makes leads to results that are just as good by the standards of anyone whose standards are worth respecting.
My working answer is therefore that it’s valuable to signal the willingness to do so (so nobody feels left out), and one effective way to signal that willingness consistently and compellingly is to precommit to actually doing it.
Sure. For example, if I want other people’s volition to be implemented, that is sufficient to justify altruism. (Not necessary, but sufficient.)
But that doesn’t justify directing an AI to look at other people’s volition to determine its target directly… as has been said elsewhere, I can simply direct an AI to look at my volition, and the extrapolation process will naturally (if CEV works at all) take other people’s volition into account.
I understand CEV. What I don’t understand is why the programmers would ask the AI for humanity’s CEV, rather than just their own CEV.
The only (sane) reason is for signalling—it’s hard to create FAI without someone else stopping you. Given a choice, however, CEV is strictly superior. If you actually do want to have FAI then FAI will be equivalent to it. But if you just think you want FAI but it turns out that, for example, FAI gets dominated by jerks in a way you didn’t expect then FAI will end up better than FAI… even from a purely altruistic perspective.
Yeah, I’ve wondered this for a while without getting any closer to an understanding.
It seems that everything that some human “really wants” (and therefore could potentially be included in the CEV target definition) is either something that, if I was sufficiently well-informed about it, I would want for that human (in which case my CEV, properly unpacked by a superintelligence, includes it for them) or is something that, no matter how well informed I was, I would not want for that human (in which case it’s not at all clear that I ought to endorse implementing it).
If CEV-humanity makes any sense at all (which I’m not sure it does), it seems that CEV-arbitrary-subset-of-humanity makes leads to results that are just as good by the standards of anyone whose standards are worth respecting.
My working answer is therefore that it’s valuable to signal the willingness to do so (so nobody feels left out), and one effective way to signal that willingness consistently and compellingly is to precommit to actually doing it.
Is this question any different from the question of why there are altruists?
Sure. For example, if I want other people’s volition to be implemented, that is sufficient to justify altruism. (Not necessary, but sufficient.)
But that doesn’t justify directing an AI to look at other people’s volition to determine its target directly… as has been said elsewhere, I can simply direct an AI to look at my volition, and the extrapolation process will naturally (if CEV works at all) take other people’s volition into account.