Why even stop at psychopaths? There are perfectly neurotypical people with strong desires for revenge-based justice, purity norms that I strongly dislike, etc. I’m not extremely confident that extrapolation will dissolve these values into deeper-order values, although my perception that intelligence in humans does at least seem to be correlated to values similar to mine is comforting in this respect.
Although really, I think this is reaching the point where we have to stop talking in terms of idealized agents with values and start thinking about how these models can be mapped to actual meat brains.
What exactly happens to all the psychopaths and people with psychopathic traits when you start extrapolating human values?
Well, under the shaky assumption that we have the ability to extrapolate in the first place, in practice what happens is that whoever controls the extrapolation sets which values are to be extrapolated, and they have a very strong incentive to put in only their own values.
By definition, no one wants to implement the CEV of humanity more than they want to implement their own CEV. But I would hope that most of the worlds impacted by the various human’s CEVs would be a pretty nice places to live.
By definition, no one wants to implement the CEV of humanity more than they want to implement their own CEV.
That depends. The more interconnected our lives become, the harder it gets to enhance the life of myself or my loved ones through highly localized improvements. Once you get up to a sufficiently high level (vaccination programs are an obvious example), helping yourself and your loved ones is easiest to accomplish by helping everyone all together, because of the ripple effects down to my loved ones’ loved ones thus having an effect on my loved ones, whom I value unto themselves.
Favoring individual volition versus a group volition could be a matter of social-graph connectedness and weighting: it could be that for a sufficiently connected individual with sufficiently strong value-weight placed on social ties, that individual will feel better about sacrificing some personal preferences to admit their connections’ values rather than simply subjecting their own close social connections to their personal volition.
But as far as your preference goes, your EV >= any other CEV. It has to be that way, tautologically. Extrapolated Volition is defined as what you would choose to do in the counter-factual scenario where you have more intelligence, knowledge, etc than you do now.
If you’re totally altruistic, it might be that your EV is the CEV of humanity, but that means that you have no preference, not that you prefer humanity’s CEV over your own. Remember, all your preferences, including the moral and altruistic ones, are included in your EV.
The notion I’m trying to express is not an entirely altruistic EV, or even a deliberately altruistic EV. Simply, this person has friends and family and such, and thus has a partially social EV; this person is at least altruistic towards close associates when it costs them nothing.
My claim, then, is that if we denote the n = number of hops from any one person to any other in the social graph of such agents:
lim_{n->0} Social Component of Personal EV = species-wide CEV
Now, there may be special cases, such as people who don’t give a shit about anyone but themselves, but the idea is that as social connectedness grows, benefitting only myself and my loved ones becomes more and more expensive and unwieldly (for instance, income inequality and guard labor already have sizable, well-studied economic costs, and that’s before we’re talking about potential improvements to the human condition from AI!) compared to just doing things that are good for everyone without regard to people’s connection to myself (they’re bound to connect through a mutual friend or relative with some low degree, after all) or social status (because again, status enforcement is expensive).
So while the total degree to which I care about other people is limited (Social Component of Personal EV ⇐ Personal EV), eventually that component should approximate the CEV of everyone reachable from me in the social graph.
The question, then, becomes whether that Social Component of my Personal EV is large enough to overwhelm some of my own personal preferences (I participate in a broader society voluntarily) or whether my personal values overwhelm my consideration of other people’s feelings (I conquer the world and crush you beneath my feet).
Why even stop at psychopaths? There are perfectly neurotypical people with strong desires for revenge-based justice, purity norms that I strongly dislike, etc. I’m not extremely confident that extrapolation will dissolve these values into deeper-order values, although my perception that intelligence in humans does at least seem to be correlated to values similar to mine is comforting in this respect.
Although really, I think this is reaching the point where we have to stop talking in terms of idealized agents with values and start thinking about how these models can be mapped to actual meat brains.
Well, under the shaky assumption that we have the ability to extrapolate in the first place, in practice what happens is that whoever controls the extrapolation sets which values are to be extrapolated, and they have a very strong incentive to put in only their own values.
By definition, no one wants to implement the CEV of humanity more than they want to implement their own CEV. But I would hope that most of the worlds impacted by the various human’s CEVs would be a pretty nice places to live.
That depends. The more interconnected our lives become, the harder it gets to enhance the life of myself or my loved ones through highly localized improvements. Once you get up to a sufficiently high level (vaccination programs are an obvious example), helping yourself and your loved ones is easiest to accomplish by helping everyone all together, because of the ripple effects down to my loved ones’ loved ones thus having an effect on my loved ones, whom I value unto themselves.
Favoring individual volition versus a group volition could be a matter of social-graph connectedness and weighting: it could be that for a sufficiently connected individual with sufficiently strong value-weight placed on social ties, that individual will feel better about sacrificing some personal preferences to admit their connections’ values rather than simply subjecting their own close social connections to their personal volition.
Then they have an altruistic EV. That’s allowed.
But as far as your preference goes, your EV >= any other CEV. It has to be that way, tautologically. Extrapolated Volition is defined as what you would choose to do in the counter-factual scenario where you have more intelligence, knowledge, etc than you do now.
If you’re totally altruistic, it might be that your EV is the CEV of humanity, but that means that you have no preference, not that you prefer humanity’s CEV over your own. Remember, all your preferences, including the moral and altruistic ones, are included in your EV.
Sorry, I don’t think I’m being clear.
The notion I’m trying to express is not an entirely altruistic EV, or even a deliberately altruistic EV. Simply, this person has friends and family and such, and thus has a partially social EV; this person is at least altruistic towards close associates when it costs them nothing.
My claim, then, is that if we denote the n = number of hops from any one person to any other in the social graph of such agents:
lim_{n->0} Social Component of Personal EV = species-wide CEV
Now, there may be special cases, such as people who don’t give a shit about anyone but themselves, but the idea is that as social connectedness grows, benefitting only myself and my loved ones becomes more and more expensive and unwieldly (for instance, income inequality and guard labor already have sizable, well-studied economic costs, and that’s before we’re talking about potential improvements to the human condition from AI!) compared to just doing things that are good for everyone without regard to people’s connection to myself (they’re bound to connect through a mutual friend or relative with some low degree, after all) or social status (because again, status enforcement is expensive).
So while the total degree to which I care about other people is limited (Social Component of Personal EV ⇐ Personal EV), eventually that component should approximate the CEV of everyone reachable from me in the social graph.
The question, then, becomes whether that Social Component of my Personal EV is large enough to overwhelm some of my own personal preferences (I participate in a broader society voluntarily) or whether my personal values overwhelm my consideration of other people’s feelings (I conquer the world and crush you beneath my feet).