Follow-up question: What is the likelihood that different modal forms of morality are fundamental?
I.e., suppose the dichotomy presented by George Lakoff’s Moral Politics turns out to describe fundamental local maxima in morality-space, which human minds can imperfectly embody.
To math this up a little, suppose the CEV of Morality Attractor 1 computes to “maximize the absolute median QALY”, while the CEV of Morality Attractor 2 computes to “maximize the the amount by which the median QALY of my in-group exceeds the median QALY of all sapiences as a whole”, and neither of those attractors have any particularly universal mathematical reason to favor them. Then, however the FAI searches the Morality domain for a CEV, it is equally likely to settle on some starry-eyed global Uplift as it is to produce nigh-infinite destitute subjects suffering indescribable anguish and despair so that a few Uplifted utility monsters have enough necks to rest their boots on.
And before anyone objects that Morality Attractor 2 is too appalling for anyone to seriously advocate, note that it has been the default behavior of most civilized societies for the majority of human history, so it must have SOMETHING going for it. Maybe Morality Attractor 1 just seems more accessible because it’s the one advocated by the mother culture that raised us, not because it’s actually what most humans tend towards as IQ/g/whatever approaches infinity.
First, I think you are significantly off-base in your contention that Morality Attractor 2 has been implemented in most civilized societies. What you see as Attractor 2 I think is better explained by Attractor 2a: “Maximize the absolute median QALY of the in-group.”, and that Clever Arguers throughout history have appealed to this desire by pointing out how much better off the in-group was, compared to a specific acceptable-target outgroup. It is, after all, much easier to provide a relative QALY surplus than an absolute QALY surplus, and our corrupted hardware is not very good at distinguishing the two. As anecdotal evidence, I consider my own morality significantly closer to 2a than 1, but definitely not similar to 2.
I would further say that it seems unlikely that the basic moral impulse is actually restricted to an arbitrary ingroup. One of those ‘perfect information’ aspects inherent in defining the output of the CEV would be knowing the life story of every person on the planet. Which is, if my knowledge of psychology is correct, basically an express ticket into the moral ingroup. This is why the single-child quarter-donation signs work, when appeals to the huge number of children suffering from don’t.
So overall, I don’t find that suggestion plausible. Someone with human-typical psychology who knew every person in existence as well as we know our friends, which is basically the postulated mind whose utility function is the CEV, would inherently value all their QALY.
nod
Follow-up question: What is the likelihood that different modal forms of morality are fundamental?
I.e., suppose the dichotomy presented by George Lakoff’s Moral Politics turns out to describe fundamental local maxima in morality-space, which human minds can imperfectly embody.
To math this up a little, suppose the CEV of Morality Attractor 1 computes to “maximize the absolute median QALY”, while the CEV of Morality Attractor 2 computes to “maximize the the amount by which the median QALY of my in-group exceeds the median QALY of all sapiences as a whole”, and neither of those attractors have any particularly universal mathematical reason to favor them. Then, however the FAI searches the Morality domain for a CEV, it is equally likely to settle on some starry-eyed global Uplift as it is to produce nigh-infinite destitute subjects suffering indescribable anguish and despair so that a few Uplifted utility monsters have enough necks to rest their boots on.
And before anyone objects that Morality Attractor 2 is too appalling for anyone to seriously advocate, note that it has been the default behavior of most civilized societies for the majority of human history, so it must have SOMETHING going for it. Maybe Morality Attractor 1 just seems more accessible because it’s the one advocated by the mother culture that raised us, not because it’s actually what most humans tend towards as IQ/g/whatever approaches infinity.
First, I think you are significantly off-base in your contention that Morality Attractor 2 has been implemented in most civilized societies. What you see as Attractor 2 I think is better explained by Attractor 2a: “Maximize the absolute median QALY of the in-group.”, and that Clever Arguers throughout history have appealed to this desire by pointing out how much better off the in-group was, compared to a specific acceptable-target outgroup. It is, after all, much easier to provide a relative QALY surplus than an absolute QALY surplus, and our corrupted hardware is not very good at distinguishing the two. As anecdotal evidence, I consider my own morality significantly closer to 2a than 1, but definitely not similar to 2.
I would further say that it seems unlikely that the basic moral impulse is actually restricted to an arbitrary ingroup. One of those ‘perfect information’ aspects inherent in defining the output of the CEV would be knowing the life story of every person on the planet. Which is, if my knowledge of psychology is correct, basically an express ticket into the moral ingroup. This is why the single-child quarter-donation signs work, when appeals to the huge number of children suffering from don’t.
So overall, I don’t find that suggestion plausible. Someone with human-typical psychology who knew every person in existence as well as we know our friends, which is basically the postulated mind whose utility function is the CEV, would inherently value all their QALY.