Be careful that you don’t say “the incentives are bad :(” as an easy out. “The incentives!” might be an infohazard, promoting a sophisticated sounding explanation for immoral behavior:
If you find yourself unable to do your job without regularly engaging in practices that clearly devalue the very science you claim to care about, and this doesn’t bother you deeply, then maybe the problem is not actually The Incentives—or at least, not The Incentives alone. Maybe the problem is You.
The lesson extends beyond science to e.g. Twitter conversations where you’re incentivized to sound snappy and confident and not change your mind publicly.
Morality means sometimes not following the incentives.
I am not saying that people who always follow the incentives are immoral. Maybe they are just lucky and their incentives happen to be aligned with doing the right thing. Too much luck is suspicious though.
I’d go a step beyond this: merely following incentives is amoral. It’s the default. In a sense, moral philosophy discusses when and how you should go against the incentives. Superhero Bias resonates with this idea, but from a different perspective.
merely following incentives is amoral. It’s the default.
Seems to me that people often miss various win/win opportunities in their life, which could make the actual human default even worse than following the incentives. On the other hand, they also miss many win/lose opportunities, maybe even more frequently.
I guess we could make a “moral behavior hierarchy” like this:
Level 1: Avoid lose/lose actions. Don’t be an idiot.
Level 2: Avoid win/lose actions. Don’t be evil.
Level 3: Identify and do win/win actions. Be a highly functioning citizen.
Level 4: Identify and do lose/win actions, if the total benefit exceeds the harm. Be a hero.
It is unfortunate that many people adopt a heuristic “avoid unusual actions”, which mostly serves them well at the first two levels, but prevents them from reaching the next two.
It is also unfortunate that some people auto-complete the pattern to “level 5: identify and do lose/win actions where the harm exceeds the benefit”, which actually reduces the total utility.
There is a certain paradox about the level 4, that the optimal moral behavior for an individual is to be a hero, but the optimal way to organize the society is so that the heroes are not necessary. (For example, by rewarding people for their heroic actions, which kinda afterwards turns them into win/win actions, which will encourage more people to do the right thing.) You will never completely eliminate the need for heroes, but you can (and should) reduce it. From this perspective, a society without heroes is unfortunate, but the society with too many heroes is probably deeply dysfunctional. We should not confuse celebrating the heroes with celebrating the environment that needs them.
Be careful that you don’t say “the incentives are bad :(” as an easy out. “The incentives!” might be an infohazard, promoting a sophisticated sounding explanation for immoral behavior:
The lesson extends beyond science to e.g. Twitter conversations where you’re incentivized to sound snappy and confident and not change your mind publicly.
Morality means sometimes not following the incentives.
I am not saying that people who always follow the incentives are immoral. Maybe they are just lucky and their incentives happen to be aligned with doing the right thing. Too much luck is suspicious though.
I’d go a step beyond this: merely following incentives is amoral. It’s the default. In a sense, moral philosophy discusses when and how you should go against the incentives. Superhero Bias resonates with this idea, but from a different perspective.
Seems to me that people often miss various win/win opportunities in their life, which could make the actual human default even worse than following the incentives. On the other hand, they also miss many win/lose opportunities, maybe even more frequently.
I guess we could make a “moral behavior hierarchy” like this:
Level 1: Avoid lose/lose actions. Don’t be an idiot.
Level 2: Avoid win/lose actions. Don’t be evil.
Level 3: Identify and do win/win actions. Be a highly functioning citizen.
Level 4: Identify and do lose/win actions, if the total benefit exceeds the harm. Be a hero.
It is unfortunate that many people adopt a heuristic “avoid unusual actions”, which mostly serves them well at the first two levels, but prevents them from reaching the next two.
It is also unfortunate that some people auto-complete the pattern to “level 5: identify and do lose/win actions where the harm exceeds the benefit”, which actually reduces the total utility.
There is a certain paradox about the level 4, that the optimal moral behavior for an individual is to be a hero, but the optimal way to organize the society is so that the heroes are not necessary. (For example, by rewarding people for their heroic actions, which kinda afterwards turns them into win/win actions, which will encourage more people to do the right thing.) You will never completely eliminate the need for heroes, but you can (and should) reduce it. From this perspective, a society without heroes is unfortunate, but the society with too many heroes is probably deeply dysfunctional. We should not confuse celebrating the heroes with celebrating the environment that needs them.
I’m fond of saying, “your ethics are only opinions until it costs you to uphold them”