I find the replaceability assumption very problematic, too. If this wasn’t LW, I would simply state the obvious an say that all sorts of evil stuff can be justified by replaceability. But this is LW, so I’ll say that replaceability is not true for reflective decision theories.
The other potential bankers aren’t using reflective decision theories. It’s really that simple.
Added: Actually, it’s even simpler: the other potential bankers have different goals. But the point about whether other people are using reflective decision theories is sometimes relevant.
If this wasn’t LW, I would simply state the obvious an say that all sorts of evil stuff can be justified by replaceability.
I’m not sure how to parse this. One possible interpretation is, “If the replaceability thesis were true, then it would follow that people should do evil things. But since people shouldn’t do evil things, it follows by modus tollens that the replaceability thesis is false.” This kind of argument could be correct depending on how the details were fleshed out, but I certainly would not call it obvious.
Another interpretation is, “Unscrupulous clever arguers could use the replaceability thesis to persuade people to do evil things.” This is more obvious, but it doesn’t seem very relevant; sufficiently bad reasoning can be used to argue for any conclusion from any set of premises.
I wasn’t trying to say anything deep, really. If the replaceability argument works for investment bankers, then it works for henchmen of an oppressive regime, too. In my country, many people actually used the replaceability argument, without the fancy name. And in hindsight people in my county agree that they shouldn’t have used the argument. So yeah, maybe it’s the modus tollens. But maybe it’s simpler than that: maybe these people misjudged being completely replaceable. In the eighties more and more people dared to say no to the Hungarian secret service, with less and less consequences.
By the way, the apparently yet-unpublished part 2 of jkaufman’s link will deal with this issue.
Well, it kind of does apply to henchmen of an opressive reigime. The classic example is Oskar Schindler: he ran munitions factories for the Nazis in order to help him smuggle Jews out of Germany (and he ran them at under capacity). Schindler is generally regarded as a hero, but that seems to be trading on precisely something like the replaceability argument. If he hadn’t done the job, someone else would have, and not only would they not have saved anybody, they would have run the factories better.
Flip the argument around for “being a banker” (or your doubtful career of choice) and it’s hard to see what changes.
Sure, I never meant to imply that the issue is clear-cut. Many of the people revealed to be informers argued that they only reported the most innocent things about the people they were tasked to spy on. Tens of thousands of books are written about such moral dilemmas. When people decide that Schindler is a hero, they seem to use a litmus test that is similar but definitely not identical to replaceability. They ask: Did he do more than what can reasonably be expected from him under his circumstances? I don’t think focusing on the replaceability part of this very complex question helps clear things up.
Okay, that’s pretty fair. I can only really claim that a replaceability argument could be used to argue that Schindler was a hero; there may be other ways of thinking about it, and those may be the ways people actually do think!
That said, I’ve found that example does sometimes make people reconsider their opinion of replaceability arguments, so it certainly appeals to something in the folk morality.
Replaceability is also not total. If you decide to be a henchman, on average you slightly increase henchman quality and reduce henchman salary. So refusing to be a henchman does cost the evil regime something.
The essay you linked to acknowledges the existence of the coordination problems I am talking about, and promises a Part 2 where it deals with them. This Part 2 is not yet published.
As Douglas_Knight shows, my comment wasn’t really well thought out. However, the idea is that a reflective decision theory agent considers the implications of the fact that whatever her decision is, similar agents will reach a similar decision. This makes them cooperate in Prisoner’s Dilemma—Tragedy of the Commons situations where “if all of us behaved so selfishly, we would be in big trouble”. The thing is sometimes called superrationality.
I find the replaceability assumption very problematic, too. If this wasn’t LW, I would simply state the obvious an say that all sorts of evil stuff can be justified by replaceability. But this is LW, so I’ll say that replaceability is not true for reflective decision theories.
The other potential bankers aren’t using reflective decision theories. It’s really that simple.
Added: Actually, it’s even simpler: the other potential bankers have different goals. But the point about whether other people are using reflective decision theories is sometimes relevant.
I’m not sure how to parse this. One possible interpretation is, “If the replaceability thesis were true, then it would follow that people should do evil things. But since people shouldn’t do evil things, it follows by modus tollens that the replaceability thesis is false.” This kind of argument could be correct depending on how the details were fleshed out, but I certainly would not call it obvious.
Another interpretation is, “Unscrupulous clever arguers could use the replaceability thesis to persuade people to do evil things.” This is more obvious, but it doesn’t seem very relevant; sufficiently bad reasoning can be used to argue for any conclusion from any set of premises.
I wasn’t trying to say anything deep, really. If the replaceability argument works for investment bankers, then it works for henchmen of an oppressive regime, too. In my country, many people actually used the replaceability argument, without the fancy name. And in hindsight people in my county agree that they shouldn’t have used the argument. So yeah, maybe it’s the modus tollens. But maybe it’s simpler than that: maybe these people misjudged being completely replaceable. In the eighties more and more people dared to say no to the Hungarian secret service, with less and less consequences.
By the way, the apparently yet-unpublished part 2 of jkaufman’s link will deal with this issue.
Well, it kind of does apply to henchmen of an opressive reigime. The classic example is Oskar Schindler: he ran munitions factories for the Nazis in order to help him smuggle Jews out of Germany (and he ran them at under capacity). Schindler is generally regarded as a hero, but that seems to be trading on precisely something like the replaceability argument. If he hadn’t done the job, someone else would have, and not only would they not have saved anybody, they would have run the factories better.
Flip the argument around for “being a banker” (or your doubtful career of choice) and it’s hard to see what changes.
Sure, I never meant to imply that the issue is clear-cut. Many of the people revealed to be informers argued that they only reported the most innocent things about the people they were tasked to spy on. Tens of thousands of books are written about such moral dilemmas. When people decide that Schindler is a hero, they seem to use a litmus test that is similar but definitely not identical to replaceability. They ask: Did he do more than what can reasonably be expected from him under his circumstances? I don’t think focusing on the replaceability part of this very complex question helps clear things up.
Okay, that’s pretty fair. I can only really claim that a replaceability argument could be used to argue that Schindler was a hero; there may be other ways of thinking about it, and those may be the ways people actually do think!
That said, I’ve found that example does sometimes make people reconsider their opinion of replaceability arguments, so it certainly appeals to something in the folk morality.
Replaceability is also not total. If you decide to be a henchman, on average you slightly increase henchman quality and reduce henchman salary. So refusing to be a henchman does cost the evil regime something.
I’m confused.
The essay you linked to acknowledges the existence of the coordination problems I am talking about, and promises a Part 2 where it deals with them. This Part 2 is not yet published.
I see. You meant the link in this post, not one of the links in the top level post (which was also me).
Can you elaborate for me, please? I don’t know what you mean (even though this is LW).
As Douglas_Knight shows, my comment wasn’t really well thought out. However, the idea is that a reflective decision theory agent considers the implications of the fact that whatever her decision is, similar agents will reach a similar decision. This makes them cooperate in Prisoner’s Dilemma—Tragedy of the Commons situations where “if all of us behaved so selfishly, we would be in big trouble”. The thing is sometimes called superrationality.
A more detailed consequentialist argument for replaceability: The replaceability effect: working in unethical industries part 1.