So I guess the takeaway is that if you care more about your status as a predictable, cooperative, and non-threatening person than about four innocent lives, don’t push the fat man.
(Also, please try to avoid sentences like “if you care about X more than innocent lives” — that comes across to me as sarcastic moral condemnation and probably tends to emotionally trigger people.)
It’s not just about what status you have, but what you actually are. You can view it as analogous to the Newcomb problem, where the predictor/Omega is able to model you accurately enough to predict if you’re going to take one or two boxes, and there’s no way to fool him into believing you’ll take one and then take both. Similarly, your behavior in one situation makes it possible to predict your behavior in other situations, at least with high statistical accuracy, and humans actually have some Omega-like abilities in this regard. If you kill the fat man, this predicts with high probability that you will be non-cooperative and threatening in other situations. This is maybe not necessarily true in the space of all possible minds, but it is true in the space of human minds—and it’s this constraint that gives humans these limited Omega-like abilities for predicting each others’ behavior.
(Of course, in real life this is further complicated by all sorts of higher-order strategies that humans employ to outsmart each other, both consciously and unconsciously. But when it comes to the fundamental issues like the conditions under which deadly violence is expected, things are usually simple and clear.)
And while these constraints may seem like evolutionary baggage that we’d best get rid of somehow, it must be recognized that they are essential for human cooperation. When dealing with a typical person, you can be confident that they’ll be cooperative and non-threatening only because you know that their mind is somewhere within the human mind-space, which means that as long as there are no red flags, cooperative and non-threatening behavior according to the usual folk-ethics is highly probable. All human social organization rests on this ability, and if humans are to self-modify into something very different, like utility-maximizers of some sort, this is a fundamental problem that must be addressed first.
Another way of saying this (I think—Vladimir_M can correct me):
You only have two choices. You can be the kind of person who kills the fat mat in order to save four other lives and kills the fat man in order to get a million dollars for yourself. Or you can be the kind of person who refuses to kill the fat man in both situations. Because of human hardware, those are your only choices.
I don’t mean to imply that the kind of person who would kill the fat man would also kill for profit. The only observation that’s necessary for my argument is that killing the fat man—by which I mean actually doing so, not merely saying you’d do so—indicates that the decision algorithms in your brain are sufficiently remote from the human standard that you can no longer be trusted to behave in normal, cooperative, and non-dangerous ways. (Which is then correctly perceived by others when they consider you scary.)
Now, to be more precise, there are actually two different issues there. The first is whether pushing the fat man is compatible with otherwise cooperative and benevolent behavior within the human mind-space. (I’d say even if it is, the latter is highly improbable given the former.) The second one is whether minds that implement some such utilitarian (or otherwise non-human) ethic could cooperate with each other the way humans are able to thanks to the mutual predictability of our constrained minds. That’s an extremely deep and complicated problem of game and decision theory, which is absolutely crucial for the future problems of artificial minds and human self-modification, but has little bearing on the contemporary problems of ideology, ethics, etc.
It seems like you can make similar arguments for virtue ethics and acausal trade.
If another agent is able to simulate you well, then it helps them to coordinate with you by knowing what you will do without communicating. When you’re not able to have a good prediction of what other people will do, it takes waaay more computation to figure out how to get what you want, and if its compatible with them getting what they want.
By making yourself easily simulated, you open yourself up to ambient control, and by not being easily simulated you’re difficult to trust. Lawful Stupid seems to happen when you have too many rules enforced too inflexibly, and often (in literature) other characters can take advantage of that really easily.
The second one is whether minds that implement some such utilitarian (or otherwise non-human) ethic could cooperate with each other the way humans are able to thanks to the mutual predictability of our constrained minds.
But we normally seem to see “one death as a tragedy, a million as a statistic” due to scope insensitivity, availability bias etc.
Why not trust that people only directly dealing with numbers are normal when they implement cold-blooded utilitarianism? Why not have many important decisions made abstractly by such people? Is wanting to make decisions this way, remote from the consequences and up a few meta-levels, a barbaric thing to advocate?
Why not trust that people only directly dealing with numbers are normal when they implement cold-blooded utilitarianism? Why not have many important decisions made abstractly by such people? Is wanting to make decisions this way, remote from the consequences and up a few meta-levels, a barbaric thing to advocate?
During the 20th century some societies have attempted to implement more-or-less that policy. The results certainly justify the adjective barbaric.
But most of the people remained relatively normal throughout. So virtue ethics needs a huge patch to approximate consequentialism.
You are providing a consequentialist argument for a base of virtue ethics plus making sure no one makes abstract decisions, but I don’t see how preventing people from making abstract decisions emerges naturally from virtue ethics at all.
I agree with your comment in one sense and was trying to imply it, as the bad results are not prevented by virtue ethics alone. On the other hand, you have provided a consequentialist argument that I think valid and was hinting towards.
Spreading this meme, even by a believing virtue ethicist, would seem to reduce the lifespan of fat men with bounties on their heads much faster than it would spare the crowds tied to the train tracks.
U: “Ooo look, a way to rationalize killing for profit!”
VE: “No no no, the message is that you shouldn’t kill the fat man in either ca-”
U: “Shush you!”
Of course, one may want to simply be the sort who tells the truth, consequences to fat men be damned.
You only have two choices. You can be the kind of person who kills the fat mat in order to save four other lives and kills the fat man in order to get a million dollars for yourself. Or you can be the kind of person who refuses to kill the fat man in both situations. Because of human hardware, those are your only choices.
No, I don’t think so. “What I actually am”, if I’m understanding Vladimir correctly, refers to the actual actions I take under various situations.
For example, if I believe I’m the sort of person who would throw the fat man under the train, but in fact I would not throw the fat man under the train, then I’ve successfully signaled to myself my status as a fat-man-under-train-thrower (I wonder if that’s an allowed construction in German), but I am not actually a fat-man-under-train-thrower.
So I guess the takeaway is that if you care more about your status as a predictable, cooperative, and non-threatening person than about four innocent lives, don’t push the fat man.
http://lesswrong.com/lw/v2/prices_or_bindings/
(Also, please try to avoid sentences like “if you care about X more than innocent lives” — that comes across to me as sarcastic moral condemnation and probably tends to emotionally trigger people.)
It’s not just about what status you have, but what you actually are. You can view it as analogous to the Newcomb problem, where the predictor/Omega is able to model you accurately enough to predict if you’re going to take one or two boxes, and there’s no way to fool him into believing you’ll take one and then take both. Similarly, your behavior in one situation makes it possible to predict your behavior in other situations, at least with high statistical accuracy, and humans actually have some Omega-like abilities in this regard. If you kill the fat man, this predicts with high probability that you will be non-cooperative and threatening in other situations. This is maybe not necessarily true in the space of all possible minds, but it is true in the space of human minds—and it’s this constraint that gives humans these limited Omega-like abilities for predicting each others’ behavior.
(Of course, in real life this is further complicated by all sorts of higher-order strategies that humans employ to outsmart each other, both consciously and unconsciously. But when it comes to the fundamental issues like the conditions under which deadly violence is expected, things are usually simple and clear.)
And while these constraints may seem like evolutionary baggage that we’d best get rid of somehow, it must be recognized that they are essential for human cooperation. When dealing with a typical person, you can be confident that they’ll be cooperative and non-threatening only because you know that their mind is somewhere within the human mind-space, which means that as long as there are no red flags, cooperative and non-threatening behavior according to the usual folk-ethics is highly probable. All human social organization rests on this ability, and if humans are to self-modify into something very different, like utility-maximizers of some sort, this is a fundamental problem that must be addressed first.
Another way of saying this (I think—Vladimir_M can correct me):
You only have two choices. You can be the kind of person who kills the fat mat in order to save four other lives and kills the fat man in order to get a million dollars for yourself. Or you can be the kind of person who refuses to kill the fat man in both situations. Because of human hardware, those are your only choices.
I don’t mean to imply that the kind of person who would kill the fat man would also kill for profit. The only observation that’s necessary for my argument is that killing the fat man—by which I mean actually doing so, not merely saying you’d do so—indicates that the decision algorithms in your brain are sufficiently remote from the human standard that you can no longer be trusted to behave in normal, cooperative, and non-dangerous ways. (Which is then correctly perceived by others when they consider you scary.)
Now, to be more precise, there are actually two different issues there. The first is whether pushing the fat man is compatible with otherwise cooperative and benevolent behavior within the human mind-space. (I’d say even if it is, the latter is highly improbable given the former.) The second one is whether minds that implement some such utilitarian (or otherwise non-human) ethic could cooperate with each other the way humans are able to thanks to the mutual predictability of our constrained minds. That’s an extremely deep and complicated problem of game and decision theory, which is absolutely crucial for the future problems of artificial minds and human self-modification, but has little bearing on the contemporary problems of ideology, ethics, etc.
It seems like you can make similar arguments for virtue ethics and acausal trade.
If another agent is able to simulate you well, then it helps them to coordinate with you by knowing what you will do without communicating. When you’re not able to have a good prediction of what other people will do, it takes waaay more computation to figure out how to get what you want, and if its compatible with them getting what they want.
By making yourself easily simulated, you open yourself up to ambient control, and by not being easily simulated you’re difficult to trust. Lawful Stupid seems to happen when you have too many rules enforced too inflexibly, and often (in literature) other characters can take advantage of that really easily.
But we normally seem to see “one death as a tragedy, a million as a statistic” due to scope insensitivity, availability bias etc.
Why not trust that people only directly dealing with numbers are normal when they implement cold-blooded utilitarianism? Why not have many important decisions made abstractly by such people? Is wanting to make decisions this way, remote from the consequences and up a few meta-levels, a barbaric thing to advocate?
During the 20th century some societies have attempted to implement more-or-less that policy. The results certainly justify the adjective barbaric.
But most of the people remained relatively normal throughout. So virtue ethics needs a huge patch to approximate consequentialism.
You are providing a consequentialist argument for a base of virtue ethics plus making sure no one makes abstract decisions, but I don’t see how preventing people from making abstract decisions emerges naturally from virtue ethics at all.
I agree with your comment in one sense and was trying to imply it, as the bad results are not prevented by virtue ethics alone. On the other hand, you have provided a consequentialist argument that I think valid and was hinting towards.
Spreading this meme, even by a believing virtue ethicist, would seem to reduce the lifespan of fat men with bounties on their heads much faster than it would spare the crowds tied to the train tracks.
U: “Ooo look, a way to rationalize killing for profit!”
VE: “No no no, the message is that you shouldn’t kill the fat man in either ca-”
U: “Shush you!”
Of course, one may want to simply be the sort who tells the truth, consequences to fat men be damned.
This seems obviously false.
Is “what you actually are” equivalent to status of yourself, to yourself?
No, I don’t think so. “What I actually am”, if I’m understanding Vladimir correctly, refers to the actual actions I take under various situations.
For example, if I believe I’m the sort of person who would throw the fat man under the train, but in fact I would not throw the fat man under the train, then I’ve successfully signaled to myself my status as a fat-man-under-train-thrower (I wonder if that’s an allowed construction in German), but I am not actually a fat-man-under-train-thrower.
http://lesswrong.com/lw/v2/prices_or_bindings/
(Also, your comment reads to me — deliberately or not — as sarcastic moral opprobrium directed at Vladimir’s position. Please try to avoid that.)