Applying these insights to the fat man/trolley problem, we see that the horrible thing about pushing the man is that it transgresses the gravest and most terrible Schelling point of all: the one that defines unprovoked deadly assault, whose violation is understood to give the other party the licence to kill the violator in self-defense. Normally, humans see such crucial Schelling points as sacrosanct. They are considered violable, if at all, only if the consequentialist scales are loaded to a far more extreme degree than in the common trolley problem formulations. Even in the latter case, the act will likely cause serious psychological damage. This is probably an artifact of additional commitment not to violate them, which may also be a safeguard against rationalizations.
Now, the utilitarian may reply that this is just human bias, an unfortunate artifact of evolutionary psychology, and we’d all be better off if people instead made decisions according to pure utilitarian calculus. However, even ignoring all the other fatal problems of utilitarianism, this view is utterly myopic. Humans are able to coordinate and cooperate because we pay respect to the Schelling points (almost) no matter what, and we can trust that others will also do so. If this were not so, you would have to be constantly alert that anyone might rob, kill, cheat, or injure you at any moment because their cost-benefit calculations have implied doing so, even if these calcualtions were in terms of the most idealistic altruistic utilitarianism. Clearly, no organized society could exist in that case: even if with unlimited computational power and perfect strategic insight you could compute that cooperation is viable, this would clearly be impractical.
It is however possible in practice for humans to evaluate each other’s personalities and figure out if others’, so to say, decision algorithms follow these constraints. Think of how people react when they realize that someone has a criminal history or sociopathic tendencies. This person is immediately perceived as creepy and dangerous, and with good reason: people realize that his decision algorithm lacks respect for the conventional Schelling points, so that normal trust and relaxed cooperation with him is impossible, and one must be on the lookout for nasty surprises. Similarly, imagine meeting someone who was in the fat man/trolley situation and who mechanically made the utilitarian decision and pushed the man without a twitch of guilt. Even the most zealous utilitarian will in practice be creeped out by such a person, even though he should theoretically perceive him as an admirable hero. (As always when it comes to ideology, people may be big on words but usually know better when their own welfare is at stake.)
(This comment is also cursory and simplified, and an alert reader will likely catch multiple imprecisions and oversimplifications. This is unfortunately unavoidable because of the complexity of the topic. However, the main point stands regardless. In particular, I haven’t addressed the all too common cases where cooperation between people breaks down and all sorts of conflict ensue. But this analysis would just reinforce the main point that cooperation critically depends on mutual recognition of near-unconditional respect for Schelling points.)
The switch example is more difficult to analyze in terms of the intuitions it evokes. I would guess that the principle of double effect captures an important aspect of what’s going on, though I’m not sure how exactly. I don’t claim to have anything close to a complete theory of human moral intuitions.
In any case, the fact that someone who flipped the switch appears much less (if at all) bad compared to someone who pushed the fat man does suggest strongly that there is some important game-theoretic issue involved, or otherwise we probably wouldn’t have evolved such an intuition (either culturally or genetically). In my view, this should be the starting point for studying these problems, with humble recognition that we are still largely ignorant about how humans actually manage to cooperate and coordinate their actions, instead of naive scoffing at how supposedly innumerate and inconsistent our intuitions are.
I unfortunately don’t get the main point :(
Could you elaborate on or at least provide a reference for how a consideration of Schelling points would suggest that we shouldn’t push the fat man?
This essay by David Friedman is probably the best treatment of the subject of Schelling points in human relations:
http://www.daviddfriedman.com/Academic/Property/Property.html
Applying these insights to the fat man/trolley problem, we see that the horrible thing about pushing the man is that it transgresses the gravest and most terrible Schelling point of all: the one that defines unprovoked deadly assault, whose violation is understood to give the other party the licence to kill the violator in self-defense. Normally, humans see such crucial Schelling points as sacrosanct. They are considered violable, if at all, only if the consequentialist scales are loaded to a far more extreme degree than in the common trolley problem formulations. Even in the latter case, the act will likely cause serious psychological damage. This is probably an artifact of additional commitment not to violate them, which may also be a safeguard against rationalizations.
Now, the utilitarian may reply that this is just human bias, an unfortunate artifact of evolutionary psychology, and we’d all be better off if people instead made decisions according to pure utilitarian calculus. However, even ignoring all the other fatal problems of utilitarianism, this view is utterly myopic. Humans are able to coordinate and cooperate because we pay respect to the Schelling points (almost) no matter what, and we can trust that others will also do so. If this were not so, you would have to be constantly alert that anyone might rob, kill, cheat, or injure you at any moment because their cost-benefit calculations have implied doing so, even if these calcualtions were in terms of the most idealistic altruistic utilitarianism. Clearly, no organized society could exist in that case: even if with unlimited computational power and perfect strategic insight you could compute that cooperation is viable, this would clearly be impractical.
It is however possible in practice for humans to evaluate each other’s personalities and figure out if others’, so to say, decision algorithms follow these constraints. Think of how people react when they realize that someone has a criminal history or sociopathic tendencies. This person is immediately perceived as creepy and dangerous, and with good reason: people realize that his decision algorithm lacks respect for the conventional Schelling points, so that normal trust and relaxed cooperation with him is impossible, and one must be on the lookout for nasty surprises. Similarly, imagine meeting someone who was in the fat man/trolley situation and who mechanically made the utilitarian decision and pushed the man without a twitch of guilt. Even the most zealous utilitarian will in practice be creeped out by such a person, even though he should theoretically perceive him as an admirable hero. (As always when it comes to ideology, people may be big on words but usually know better when their own welfare is at stake.)
(This comment is also cursory and simplified, and an alert reader will likely catch multiple imprecisions and oversimplifications. This is unfortunately unavoidable because of the complexity of the topic. However, the main point stands regardless. In particular, I haven’t addressed the all too common cases where cooperation between people breaks down and all sorts of conflict ensue. But this analysis would just reinforce the main point that cooperation critically depends on mutual recognition of near-unconditional respect for Schelling points.)
Can you explain why this analysis renders directing away from the five and toward the one permissible?
The switch example is more difficult to analyze in terms of the intuitions it evokes. I would guess that the principle of double effect captures an important aspect of what’s going on, though I’m not sure how exactly. I don’t claim to have anything close to a complete theory of human moral intuitions.
In any case, the fact that someone who flipped the switch appears much less (if at all) bad compared to someone who pushed the fat man does suggest strongly that there is some important game-theoretic issue involved, or otherwise we probably wouldn’t have evolved such an intuition (either culturally or genetically). In my view, this should be the starting point for studying these problems, with humble recognition that we are still largely ignorant about how humans actually manage to cooperate and coordinate their actions, instead of naive scoffing at how supposedly innumerate and inconsistent our intuitions are.
Thanks! That makes sense.