What this measures is holding socially acceptable beliefs and behaving in a socially acceptable way. I don’t know why he thinks this is what morality is. After all, the entire village collectively and joyfully cooperates in burning the witch.
Another type of scenario involves minorities. Imagine, for instance, that 98% of the players are unfailingly nice to each other, but unfailingly cruel to the remaining 2% (who they can recognize, let’s say, by their long noses or darker skin—some trivial feature like that). Meanwhile, the put-upon 2% return the favor by being nice to each other and mean to the 98%. Who, in this scenario, is moral, and who’s immoral? The mathematical verdict of both eigenmoses and eigenjesus is unequivocal: the 98% are almost perfectly good, while the 2% are almost perfectly evil. After all, the 98% are nice to almost everyone, while the 2% are mean to those who are nice to almost everyone, and nice only to a tiny minority who are mean to almost everyone. Of course, for much of human history, this is precisely how morality worked, in many people’s minds. But I dare say it’s a result that would make moderns uncomfortable.
There’s a crucial observation that I took for granted in the post but shouldn’t have, so let me now make it explicit. The observation is this:
No system for aggregating preferences whatsoever—neither direct democracy, nor representative democracy, nor eigendemocracy, nor anything else—can possibly deal with the “Nazi Germany problem,” wherein basically an entire society’s value system becomes inverted to the point where evil is good and good evil.
By the way, this is also related to the argument in “Well-Kept Gardens Die By Pacifism”. When we design a system for moderating a web community, we are choosing between “order” and “chaos”, not between “good” and “evil”.
We can move the power to moderator, to some inner circle of users, to most active users, even to users with most sockpuppets, but we can’t just move it to “good”. We can choose which kind of people or which kind of behavior gets the most power, but we can’t choose that the power will magically disappear if they try to abuse it; because any rule designed to prevent abuse can also be abused. The values have to come from outside of the voting system; from the humans who use it. So at the end, the only reasonable choice is to design the system to preserve the existing power, whatever it is—allowing change only when it is initiated by the currently existing power—because the only alternative is to let forces from outside of the garden optimize for their values, again, whatever they are, not only the “good” ones. And yes, if the web community had a horrible values at the beginning, the proper moderating system will preserve them. That’s not bug, that’s a side-effect of a feature. (Luckily, on the web, you have the easy option of leaving the community.)
In this sense, we have to realize that the eigen-whatever system proposed in the article, if designed correctly (how to do this specifically is still open to discussion), would capture something like “the applause lights of the majority of the influential people”, or something similar. If the “majority of the influential people” are evil, or just plain stupid, the eigen-result can easily contain evil or stupidity. It almost certainly contains religion and other irrationality. At best, this system is a useful tool to see what the “majority of influential people” think is morality (as V_V said), which itself is a very nice result for a mathematical equation, but I wouldn’t feel immoral for disagreeing with in at some specific points. Also, it misses the “extrapolated” part of the CEV; for example, if people’s moral opinions are based on incorrect or confused beliefs, the result will contain morality based on incorrect beliefs, so it could give you a recommendation to do both X and Y, where X and Y are contradictory.
Well yes, and attempting to group all actual or possible individuals into one tribe is a major mistake, one that I think should be given a name. Well, as it turns out, the name I was already going to give it is at least partially in use: False Universalism.
Ethics ought to include some kind of reasoning for determining when some bit of universalism (some universalization of a maxim, in the Kantian or Timeless sense, or some value cohering, in the CEV sense) has become False Universalism, so that the groups or individuals who diverge from each other to the point of incompatibility can be handled as conflicting, rather than simply having the ethical algorithm return the answer that one or the other is Right and the other is Wrong and the Wrong shall be corrected until they follow the values of the Right.
“Handled as conflicting” seems to either mean “all-out war” or at best “temporary putting off of all-out war until we’ve used all the atoms on our side of the universe”.
If the two sides shared your desire to be symmetrically peaceful with other sides whose only point of similarity with them was the desire to be symmetrically peaceful with other sides whose… then Universalism isn’t false. That’s its minimal case.
And if it does fail, it seems counterproductive for you to point that out to us, because while we’re happily and deludedly trying to apply it, we’re not genociding each other all over your lawn.
Sorry, when I said “False Universalism”, I meant things like, “one group wants to have kings, and another wants parliamentary democracy”. Or “one group wants chocolate, and the other wants vanilla”. Common moral algorithms seem to simply assume that the majority wins, so if the majority wants chocolate, everyone gets chocolate. Moral constructionism gets around this by saying: values may not be universal, but we can come to game-theoretically sound agreements (even if they’re only Timelessly sound, like Rawls’ Theory of Justice) on how to handle the disagreements productively, thus wasting fewer resources on fighting each other when we could be spending them on Fun.
Basically, I think the correct moral algorithm is: use a constructionist algorithm to cluster people into groups who can then use realist universalisms internally.
If we’re aggregating cooperation rather than aggregating values, we certainly can create a system that distinguishes between societies that apply an extreme level of noncooperation (i.e. killing) to larger groups of people than other societies, and that uses our own definition of noncooperation rather than what the Nazi values judge as noncooperation.
That’s not to say you couldn’t still find tricky example societies where the system evaluation isn’t doing what we want, I just mean to encourage further improvement to cover moral behaviour towards and from hated minorities, and in actual Nazi Germany.
But his own scheme isn’t the aggregation of arbitrary values, it’s based on rewarding co operation.
Perhaps in-group problems could be fixed with an eigenSinger algorithm that gives extra points to those who cop operate with people they have not cooperated with before, ie widening the circle.
The burning of a witch is a rightful celebration because the community is rid of a serious danger and the spectacle is a welcome diversion from everyday drudgery :-/
In retrospect, so much of my awareness of actual historical European and American witch-scares is so heavily colored by modern politics and fiction — as, I suspect, most folks’ is — that I think we would probably be ill-advised to draw many conclusions from it.
Well, we can make it easier. The age of lynchings and the age of photography intersected. Here, take a look at these two photographs and tell me what the mood of the crowd is.
Warning: persons of sensitive nervous disposition should not click on the links as there will be gruesomeness involved.
I’m broadly sympathetic to your point that Aaronson’s algorithm is measuring something like conformity or group consensuses rather than morality, but the specific example you picked to garnish that point is dodgy. From Randall Collins’s Violence: A Micro-sociological Theory, pages 425-426:
But in fact, as I have tried to show throughout, violent
confrontations generate their own situational emotions, above all ten-
sion and fear; and we see this on the faces and in the body language of
most of the crowd during these events. The demonstrative extremists are
not expressing an emotion that we can safely infer, on the basis of the
evidence, as generally existing in the crowd; they are another specialized
minority (different as well from the minority of violent activists) who
have found their own emotional niche in the context of the crowd. And
in fact the bulk of the crowd does not usually follow them, even when
it is safe to do so because there is no opponent present but only a dead
body or a captured building; the demonstrative extremists are usually
separate from the rest of the crowd.
Some evidence bearing on this point comes from photos of lynchings
in the American South and West in the period 1870-1935 (Allen 2000).
Most of these photos—and the ones that are most revealing on this
point—were taken several hours in the aftermath of the actual violence,
or the following day. Individuals offering acts of gratuitous insult to dead
bodies are safely disconnected from the actual commission of the violence.
In a photo (Allen 2000: plate 93, not shown here) we see two white men
standing beside the body of a black man hung from a tree, one poking
the body with a stick, the other punching it. In the background are four
other white men looking on. In another photo (Allen 2000: plate 25, not
shown) a young man leans nonchalantly with his eyes closed against the
post from which the charred body of a black man, lynched the previous
night, is hanging; the nineteen other faces visible in the crowd are somber.
This is the usual pattern in all the photos: a few are the demonstrative
extremists; most of the others are somber, serious, awed, or uncomfort-
able in the presence of death. [I skip a paragraph here.]
But the mood of demonstrative extremists in the aftermath contrasts
with the emotions that are displayed during the actual lynching itself. In
a rare set of photos, we see a lynching while it is going on: a black man
showing the welts of whipping on his back stands in a wagon shortly
before he is hung; his executioners stare at his face with hard, hostile
stares (Allen 2000: plates 42 and 43; not shown here). There is no clown-
ing, no expression of joy; these are the frontline activist few, engaged in
an angry, domineering stare-down. A violent confrontation itself is tense;
even when one side has the upper hand (the usual formula for successful
violence) it is compelling, focused into the business at hand, unable to
ironicize it.
Thus the demonstrative extremists, operating safely in the aftermath,
show something else: an attempt to raise themselves from the bulk of the
crowd, the back line of merely nominal supporters of the violence, and
to raise their status by closer connection with the violent action that had
galvanized the attention of the group. By putting themselves close to the
dead body, engaging in gratuitous acts of insult upon it, they put them-
selves closer to the center of the attention space.⁹ The most blatant expres-
sion of joy among Allen’s (2000) lynching photos is in plate 97 (not
shown), which shows two well-dressed young men grinning (rather mirth-
lessly) at the camera while viewing a black body being burned. This was
not in the heat of the action, however, but in the aftermath; the victim,
accused of molesting a white girl, had already been hung from a lamppost
and riddled with bullets. These cheery demonstrative extremists are prob-
ably proud of themselves for standing so close to the workman in grimy
overalls who is tending the fire; everyone else in the photo is farther away,
and the twenty-nine other visible faces range from somber to apprehen-
sive.
The last photo Collins refers to (“plate 97”) sounds like it’s your photo 2, and Collins’s interpretation of it seems to me to fit better than the one you imply. I wouldn’t go as far as to say that all of the “twenty-nine other visible faces range from somber to apprehensive”, as a few of the people in the photo are evidently just trying to get a closer look, and a boy near the front seems more intrigued than anything else. Nonetheless, the crowd as a whole doesn’t appear joyful or celebratory to me.
Turning to photo 1, I see perhaps ten spectators’ faces clearly enough to make a guess at what they’re expressing. Going from left to right, I see (1) thick-eyebrowed man in cap in background who looks as if he’s whistling while thinking hard about something; (2) foreground man in white shirt with no tie with neutral-ish expression, but maybe happy; (3) man in middle distance with shorter man in front and a boater-wearer behind, the former gazing off to the photographer’s right with a worried/pensive look; (4) smirking man in tie; (5) foreground woman in dress with irregular spots who looks surprised/wary; (6) another woman behind her with open mouth, who might be amused or surprised or scandalized; (7) cluster of three foreground women, where the nearest one’s face is too blurry for me to interpret, but the two further back both look apprehensive; (8) foreground man with moustache, pointing as he stares intently; and (9) man at right edge with left eye not visible in photo, looking not especially happily to the photographer’s right. As in photo 2, although there are happy-looking people present, they are a minority.
I don’t believe it’s accurate to use these scenes as examples of rightful celebration or joy. fubarobfusco’s warning to exercise caution in how we read these photos may be well-advised.
Yes, I know these photos were analyzed quite substantially, but my point is really simple—it’s that the lynchings (and the witch burnings before them) were culturally normal. The were intense events and, of course, brought out a range of emotions, not just joy, but all I’m trying to say is that the majority of people did not see them as something to be ashamed of. It was OK, it was fine, it was moral.
I’m well aware of such things. My question was aimed at the phrase “the entire village” — which is a claim of unanimity. My point was that members of the village who disapproved would have a strong incentive not to express their disapproval.
Also, American lynchings (of the period you refer to) and witch-hunts are not really the same thing, socially speaking — in part because of the supernatural corruption implied by a claim of witchcraft.
Witch-hunts are an ongoing thing, by the way, in a number of parts of the world. In some cases, economic motives are pretty clear; in others, it seems pretty clearly a matter of superstitious fear that isn’t really comparable to the self-evident political or sexual-political motives behind American racial lynchings.
What this measures is holding socially acceptable beliefs and behaving in a socially acceptable way. I don’t know why he thinks this is what morality is. After all, the entire village collectively and joyfully cooperates in burning the witch.
Scott remarks on this himself:
Scott Says:
By the way, this is also related to the argument in “Well-Kept Gardens Die By Pacifism”. When we design a system for moderating a web community, we are choosing between “order” and “chaos”, not between “good” and “evil”.
We can move the power to moderator, to some inner circle of users, to most active users, even to users with most sockpuppets, but we can’t just move it to “good”. We can choose which kind of people or which kind of behavior gets the most power, but we can’t choose that the power will magically disappear if they try to abuse it; because any rule designed to prevent abuse can also be abused. The values have to come from outside of the voting system; from the humans who use it. So at the end, the only reasonable choice is to design the system to preserve the existing power, whatever it is—allowing change only when it is initiated by the currently existing power—because the only alternative is to let forces from outside of the garden optimize for their values, again, whatever they are, not only the “good” ones. And yes, if the web community had a horrible values at the beginning, the proper moderating system will preserve them. That’s not bug, that’s a side-effect of a feature. (Luckily, on the web, you have the easy option of leaving the community.)
In this sense, we have to realize that the eigen-whatever system proposed in the article, if designed correctly (how to do this specifically is still open to discussion), would capture something like “the applause lights of the majority of the influential people”, or something similar. If the “majority of the influential people” are evil, or just plain stupid, the eigen-result can easily contain evil or stupidity. It almost certainly contains religion and other irrationality. At best, this system is a useful tool to see what the “majority of influential people” think is morality (as V_V said), which itself is a very nice result for a mathematical equation, but I wouldn’t feel immoral for disagreeing with in at some specific points. Also, it misses the “extrapolated” part of the CEV; for example, if people’s moral opinions are based on incorrect or confused beliefs, the result will contain morality based on incorrect beliefs, so it could give you a recommendation to do both X and Y, where X and Y are contradictory.
Well yes, and attempting to group all actual or possible individuals into one tribe is a major mistake, one that I think should be given a name. Well, as it turns out, the name I was already going to give it is at least partially in use: False Universalism.
Ethics ought to include some kind of reasoning for determining when some bit of universalism (some universalization of a maxim, in the Kantian or Timeless sense, or some value cohering, in the CEV sense) has become False Universalism, so that the groups or individuals who diverge from each other to the point of incompatibility can be handled as conflicting, rather than simply having the ethical algorithm return the answer that one or the other is Right and the other is Wrong and the Wrong shall be corrected until they follow the values of the Right.
“Handled as conflicting” seems to either mean “all-out war” or at best “temporary putting off of all-out war until we’ve used all the atoms on our side of the universe”.
If the two sides shared your desire to be symmetrically peaceful with other sides whose only point of similarity with them was the desire to be symmetrically peaceful with other sides whose… then Universalism isn’t false. That’s its minimal case.
And if it does fail, it seems counterproductive for you to point that out to us, because while we’re happily and deludedly trying to apply it, we’re not genociding each other all over your lawn.
Sorry, when I said “False Universalism”, I meant things like, “one group wants to have kings, and another wants parliamentary democracy”. Or “one group wants chocolate, and the other wants vanilla”. Common moral algorithms seem to simply assume that the majority wins, so if the majority wants chocolate, everyone gets chocolate. Moral constructionism gets around this by saying: values may not be universal, but we can come to game-theoretically sound agreements (even if they’re only Timelessly sound, like Rawls’ Theory of Justice) on how to handle the disagreements productively, thus wasting fewer resources on fighting each other when we could be spending them on Fun.
Basically, I think the correct moral algorithm is: use a constructionist algorithm to cluster people into groups who can then use realist universalisms internally.
If we’re aggregating cooperation rather than aggregating values, we certainly can create a system that distinguishes between societies that apply an extreme level of noncooperation (i.e. killing) to larger groups of people than other societies, and that uses our own definition of noncooperation rather than what the Nazi values judge as noncooperation.
That’s not to say you couldn’t still find tricky example societies where the system evaluation isn’t doing what we want, I just mean to encourage further improvement to cover moral behaviour towards and from hated minorities, and in actual Nazi Germany.
But his own scheme isn’t the aggregation of arbitrary values, it’s based on rewarding co operation.
Perhaps in-group problems could be fixed with an eigenSinger algorithm that gives extra points to those who cop operate with people they have not cooperated with before, ie widening the circle.
Collectively, perhaps; but joyfully? Speaking out in defense of the witch risks being labeled a witch yourself.
The burning of a witch is a rightful celebration because the community is rid of a serious danger and the spectacle is a welcome diversion from everyday drudgery :-/
In retrospect, so much of my awareness of actual historical European and American witch-scares is so heavily colored by modern politics and fiction — as, I suspect, most folks’ is — that I think we would probably be ill-advised to draw many conclusions from it.
Well, we can make it easier. The age of lynchings and the age of photography intersected. Here, take a look at these two photographs and tell me what the mood of the crowd is.
Warning: persons of sensitive nervous disposition should not click on the links as there will be gruesomeness involved.
Photo 1 Photo 2
I’m broadly sympathetic to your point that Aaronson’s algorithm is measuring something like conformity or group consensuses rather than morality, but the specific example you picked to garnish that point is dodgy. From Randall Collins’s Violence: A Micro-sociological Theory, pages 425-426:
The last photo Collins refers to (“plate 97”) sounds like it’s your photo 2, and Collins’s interpretation of it seems to me to fit better than the one you imply. I wouldn’t go as far as to say that all of the “twenty-nine other visible faces range from somber to apprehensive”, as a few of the people in the photo are evidently just trying to get a closer look, and a boy near the front seems more intrigued than anything else. Nonetheless, the crowd as a whole doesn’t appear joyful or celebratory to me.
Turning to photo 1, I see perhaps ten spectators’ faces clearly enough to make a guess at what they’re expressing. Going from left to right, I see (1) thick-eyebrowed man in cap in background who looks as if he’s whistling while thinking hard about something; (2) foreground man in white shirt with no tie with neutral-ish expression, but maybe happy; (3) man in middle distance with shorter man in front and a boater-wearer behind, the former gazing off to the photographer’s right with a worried/pensive look; (4) smirking man in tie; (5) foreground woman in dress with irregular spots who looks surprised/wary; (6) another woman behind her with open mouth, who might be amused or surprised or scandalized; (7) cluster of three foreground women, where the nearest one’s face is too blurry for me to interpret, but the two further back both look apprehensive; (8) foreground man with moustache, pointing as he stares intently; and (9) man at right edge with left eye not visible in photo, looking not especially happily to the photographer’s right. As in photo 2, although there are happy-looking people present, they are a minority.
I don’t believe it’s accurate to use these scenes as examples of rightful celebration or joy. fubarobfusco’s warning to exercise caution in how we read these photos may be well-advised.
Yes, I know these photos were analyzed quite substantially, but my point is really simple—it’s that the lynchings (and the witch burnings before them) were culturally normal. The were intense events and, of course, brought out a range of emotions, not just joy, but all I’m trying to say is that the majority of people did not see them as something to be ashamed of. It was OK, it was fine, it was moral.
I’m well aware of such things. My question was aimed at the phrase “the entire village” — which is a claim of unanimity. My point was that members of the village who disapproved would have a strong incentive not to express their disapproval.
Also, American lynchings (of the period you refer to) and witch-hunts are not really the same thing, socially speaking — in part because of the supernatural corruption implied by a claim of witchcraft.
Witch-hunts are an ongoing thing, by the way, in a number of parts of the world. In some cases, economic motives are pretty clear; in others, it seems pretty clearly a matter of superstitious fear that isn’t really comparable to the self-evident political or sexual-political motives behind American racial lynchings.
Recall the top-level post.
Sure there will be deviants who disapprove. They are, clearly, very bad people.
They are expressions of the morality of the majority which is good by definition, isn’t it?