Rationalists are fond of saying that the problems of the world are not from people being evil, but instead a result of the incentives of our system, which are such that this bad outcome is an equilibrium. There’s a weaker thesis here that I agree with, but otherwise I don’t think this argument actually follows.
In game theory, an equilibrium is determined by both the setup of the game, and by the payoffs for each player. The payoffs are basically the values of the players in the game—their utility functions. In other words, you get different equilibria if players adopt different values.
Problems like homelessness are caused by zoning laws, yes, but they’re also caused by people being selfish. Why? Because lots of people could just voluntarily donate their wealth to help homeless people. Anyone with a second house could decide to give it away. Those with spare rooms could simply rent them out for free. There are no laws saying you must spend your money on yourself.
A simple economic model would predict that if we redistributed everyone’s extra housing, then this would reduce the incentive to create new housing. But look closer at the assumptions in that economic model. We say that the incentives to build new housing are reduced because few people will pay to build a house if they don’t get to live in it or sell it to someone else. That’s another way of assuming that people value their own consumption more than that of others—another way of saying that people are selfish.
More fundamentally, what it means for something to be an incentive is that it helps people get what they want. Incentives, therefore, are determined by people’s values; they are not separate from them. A society of saints would have different equilibria than a society of sinners, even if both are playing the same game. So, it really is true that lots of problems are caused by people being bad.
Of course, there’s an important sense in which rationalists are probably right. Assume that we can change the system but we can’t change people’s values. Then, pragmatically, the best thing would be to change the system, rather than fruitlessly try to change people’s values.
Yet it must be emphasized that this hypothesis is contingent on the relative tractability of either intervention. If it becomes clear that we can genuinely make people less selfish, then that might be a good thing to try.
My main issue with attempts to redesign society in order to make people less selfish or more cooperative is that you can’t actually change people’s innate preferences by very much. The most we can reasonably hope for is to create a system in which people’s selfish values are channeled to produce social good. That’s not to say it wouldn’t be nice if we could change people’s innate preferences. But we can’t (yet).
(Note that I wrote this as a partial response to jimrandomh’s shortform post, but the sentiment I’m responding to is more general than his exact claim.)
The connection between “doing good” and “making a sacrifice” is so strong that people need to be reminded that “win/win” is also a thing. The bad guys typically do whatever is best for them, which often involves hurting others (because some resources are limited). The good guys exercise restraint.
This is complicated because there is also the issue of short-term and long-term thinking. Sometimes the bad guys do things that benefit them in short term, but contribute to their fall in long term; while the good guys increase their long-term gains by strategically giving up on some short-term temptations. But it is a just-world fallacy to assume that things always end up this way. Sometimes the badguys murder millions, and then they live happily to old age. Sometimes the good guys get punished and laughed at, and then they die in despair.
How could “good” even have evolved, given that “sacrifice” seems by definition incompatible with “maximizing fitness”?
being good to your relatives promotes your genes.
reciprocal goodness can be an advantage to both players.
doing good—precisely because it is a sacrifice—can become a signal of abundance, which makes other humans want to be my allies or mates.
people reward good and punish evil in others, because it is in their selfish interest to live among good people.
The problems caused by the evolutionary origin of goodness are also well-known: people are more likely to be good towards their neighbors who can reciprocate or towards potential sexual partners, and they are more likely to do good when they have an audience who approves of it… and less likely to do good to low-status people who can’t reciprocate, or when their activities are anonymous. (Steals money from pension funds, polutes the environment, then donates millions to a prestigious university.)
I assume that most people are “instinctively good”, that is that they kinda want to be good, but they simply follow their instincts, and don’t reflect much on them (other than rationalizing that following their instinct was good, or at least a necessary evil). Their behavior can be changed by things that affect their instincts—the archetypal example is the belief in an omniscient judging God, i.e. a powerful audience who sees all behavior, and rewards/punishes according to social norms (so now the only problem is how to make those social norms actually good). I am afraid that this ship has sailed, and that we do not really have a good replacement—any non-omniscient judge can be deceived, and any reward mechanism will be Goodharted. Another problem is that by trying to make society more tolerant and more governed by law, we also take away people’s ability to punish evil… as long as the evil takes care to only do evil acts that are technically legal, or when there is not enough legal evidence of wrongdoing.
Assuming we have a group of saints (who have the same values, and who trust each other to be saints), I am not even sure what would be the best strategy for them. Probably to cooperate with each other a lot, because there is no risk of being stabbed in the back. Try to find other saints, test them, and then admit them to the group. Notice good acts among non-saints and reward them somehow—maybe in form of lottery, when most good acts only get a “thank you”, but one in a million gets a million-dollar reward. (People overestimate their chances in lottery. This would lead them to overestimate how likely a good act is to be rewarded, which would make them do more good.) The obvious problem with rewarding good acts is that it rewards visibility; perhaps there should be a special rewards for good acts that were unlikely to get noticed. The good acts should get a social reward, i.e. telling other people about the good act and how someone was impressed.
(The sad thing is that given that we live in a clickbait society, it would not take much time until someone would publish an article about how X-ist the saints are, because the proportion of Y’s they rewarded for good deeds is not the same as the proportion of Y’s in the society. Also, this specific person rewarded for this specific good deed also happens to hold some problematic opinions, does this mean that the saints secretly support the opinion, too?)
I sometimes like to imagine a soft version of karma, like if people would be free to associate with people who are like them, then the good people would associate with other good people, the bad people would associate with other bad people, and then the bad people would suffer (because surrounded by bad people), and the good people would live nice lives (because surrounded by good people). The problem with this vision is that people are not so free to choose their neighbors (coordination is hard, moving is expensive), and also that the good people who suck at judging other people’s goodness would suffer. Not sure what is the right approach here, other than perhaps we should become a bit more judgmental, because it seems the pendulum has swung too much in the direction that you are not even allowed to criticize [an obviously horrible thing] out of concern that some culture might routinely [do the horrible thing], which would get you called out as intolerant, which is a sin much worse than [doing the horrible thing]. I’d like people to get some self-respect and say “hey, these are my values, if you disagree, fuck off”. But this of course assumes that the people who disagree actually have a place to go. Another problem is that you cannot build an archipelago, if the land is scarce, and your solution to conflicts is to walk away.
(Also, a fraction of people are literally psychopaths, so even if we devised a set of nudges to make most people behave good, it would not apply to everyone. To make someone behave good out of mere rational self-interest, they would have to believe that almost all evil deeds get detected and punished, which is very difficult to achieve.)
I usually associate things like “being evil” more with something like “part of my payoff matrix has a negative coefficient on your payoff matrix”. I.e. actively wanting to hurt people and taking inherent interest in making them worse off. Selfishness feels pretty different from being evil emotionally, at least to me.
Judgement of evil follows the same pressures as evil itself. Selfishness feels different from sadism to you, at least in part because it’s easier to find cooperative paths with selfishness. And this question really does come down to “when should I cooperate vs defect”.
If your well-being has exactly zero value in my preference function, that literally means that I would kill you in a dark alley if I believed there was zero chance of being punished, because there is a chance you might have some money that I could take. I would call that “evil”, too.
You can’t hypothesize zeros and get anywhere. MANY MANY psychopaths exist, and very few of them find it more effective to murder people for spare change than to further their ends in other ways. They may not care about you, but your atoms are useful to them in their current configuration.
They may not care about you, but your atoms are useful to them in their current configuration.
There are ways of hurting people other than stabbing them, I just used a simple example.
I think there is a confusion about what exactly “selfish” means, and I blame Ayn Rand for it. The heroes in her novels are given the label “selfish” because they do not care about possibilities to actively do something good for other people unless there is also some profit for them (which is what a person with zero value for others in their preference function would do), but at the same time they avoid actively harming other people in ways that could bring them some profit (which is not what a perfectly selfish person would do).
As a result, we get quite unrealistic characters who on one hand are described as rational profit maximizers who don’t care about others (except instrumentally), but on the other hand they follow an independently reinvented deontological framework that seems like designed by someone who actually cares about other people but is in deep denial about it (i.e. Ayn Rand).
A truly selfish person (someone who truly does not care about others) would hurt others in situations where doing so is profitable (including second-order effects). A truly selfish person would not arbitrarily invent a deontological code against hurting other people, because such code is merely a rationalization invented by someone who already has an emotional reason not to hurt other people but wants to pretend that instead this is a logical conclusion derived from first principles.
Interacting with a psychopath with likely get you hurt. It will likely not get you killed, because some other way of hurting you has a better risk:benefit profile. Perhaps the most profitable way is to scam you of some money and use you to get introduced to your friends. Only once in a while a situation will arise when raping someone is sufficiently safe, or killing someone is extremely profitable, e.g. because that person stands in a way of a grand business.
I’m not sure what our disagreement actually is—I agree with your summary of Ayn Rand, I agree that there are lots of ways to hurt people without stabbing. I’m not sure you’re claiming this, but I think that failure to help is selfish too, though I’m not sure it’s comparable with active harm.
It may be that I’m reacting badly to the use of “truly selfish”—I fear a motte-and-bailey argument is coming, where we define it loosely, and then categorize actions inconsistently as “truly selfish” only in extremes, but then try to define policy to cover far more things.
I think we’re agreed that the world contains a range of motivated behaviors, from sadistic psychopaths (who have NEGATIVE nonzero terms for others’ happiness) to saints (whose utility functions weight very heavily toward other’s happiness over their own). I don’t know if we agree that “second-order effect” very often dominate the observed behaviors over most of this range. I hope we agree that almost everyone changes their behavior to some extent based on visible incentives.
I still disagree with your post that a coefficient of 0 for you in someone’s mind implies murder for pocket change. And I disagree with the implication that murder for pocket change is impossible even if the coefficient is above 0 - circumstances matter more than innate utility function.
To the OP’s point, it’s hard to know how to accomplish “make people less selfish”, but “make the environment more conducive to positive-sum choices so selfish people take cooperative actions” is quite feasible.
I still disagree with your post that a coefficient of 0 for you in someone’s mind implies murder for pocket change.
I believe this is exactly what it means, unless there is a chance of punishment or being hurt by victim’s self-defense or a chance of better alternative interaction with given person. Do you assume that there is always a more profitable interaction? (What if the target says “hey, I just realized that you are a psychopath, and I do not want to interact with you anymore”, and they mean it.)
Could you please list the pros and cons of deciding whether to murder a stranger who refuses to interact with you, if there is zero risk of being punished, from the perspective of a psychopath? As I see it, the “might get some pocket change” in the pro column is the only nonzero item in this model.
unless there is a chance of punishment or being hurt by victim’s self-defense or a chance of better alternative interaction with given person.
There always is that chance. That’s mostly our disagreement. Using real-world illustrations (murder) for motivational models (utility) really needs to acknowledge the uncertainty and variability, which the vast majority of the time “adds up to normal”. There really aren’t that many murders among strangers. And there are a fair number of people who don’t value others’ very highly.
Yes, I would make this distinction too. Yet, I submit that few people actually believe, or even say they believe, that the main problems in the world are caused by people being gratuitously or sadistically evil. There are some problems that people would explain this way: violent crime comes to mind. But I don’t think the evil hypothesis is the most common explanation given by non-rationalists for why we have, say, homelessness and poverty.
That is to say that, insofar as the common rationalist refrain of “problems are caused by incentives dammit, not evil people” refers to an actual argument people generally give, it’s probably referring to the argument that people are selfish and greedy. And in that sense, the rationalists and non-rationalists are right: it’s both the system and the actors within it.
Rationalists are fond of saying that the problems of the world are not from people being evil, but instead a result of the incentives of our system, which are such that this bad outcome is an equilibrium. There’s a weaker thesis here that I agree with, but otherwise I don’t think this argument actually follows.
In game theory, an equilibrium is determined by both the setup of the game, and by the payoffs for each player. The payoffs are basically the values of the players in the game—their utility functions. In other words, you get different equilibria if players adopt different values.
Problems like homelessness are caused by zoning laws, yes, but they’re also caused by people being selfish. Why? Because lots of people could just voluntarily donate their wealth to help homeless people. Anyone with a second house could decide to give it away. Those with spare rooms could simply rent them out for free. There are no laws saying you must spend your money on yourself.
A simple economic model would predict that if we redistributed everyone’s extra housing, then this would reduce the incentive to create new housing. But look closer at the assumptions in that economic model. We say that the incentives to build new housing are reduced because few people will pay to build a house if they don’t get to live in it or sell it to someone else. That’s another way of assuming that people value their own consumption more than that of others—another way of saying that people are selfish.
More fundamentally, what it means for something to be an incentive is that it helps people get what they want. Incentives, therefore, are determined by people’s values; they are not separate from them. A society of saints would have different equilibria than a society of sinners, even if both are playing the same game. So, it really is true that lots of problems are caused by people being bad.
Of course, there’s an important sense in which rationalists are probably right. Assume that we can change the system but we can’t change people’s values. Then, pragmatically, the best thing would be to change the system, rather than fruitlessly try to change people’s values.
Yet it must be emphasized that this hypothesis is contingent on the relative tractability of either intervention. If it becomes clear that we can genuinely make people less selfish, then that might be a good thing to try.
My main issue with attempts to redesign society in order to make people less selfish or more cooperative is that you can’t actually change people’s innate preferences by very much. The most we can reasonably hope for is to create a system in which people’s selfish values are channeled to produce social good. That’s not to say it wouldn’t be nice if we could change people’s innate preferences. But we can’t (yet).
(Note that I wrote this as a partial response to jimrandomh’s shortform post, but the sentiment I’m responding to is more general than his exact claim.)
The connection between “doing good” and “making a sacrifice” is so strong that people need to be reminded that “win/win” is also a thing. The bad guys typically do whatever is best for them, which often involves hurting others (because some resources are limited). The good guys exercise restraint.
This is complicated because there is also the issue of short-term and long-term thinking. Sometimes the bad guys do things that benefit them in short term, but contribute to their fall in long term; while the good guys increase their long-term gains by strategically giving up on some short-term temptations. But it is a just-world fallacy to assume that things always end up this way. Sometimes the bad guys murder millions, and then they live happily to old age. Sometimes the good guys get punished and laughed at, and then they die in despair.
How could “good” even have evolved, given that “sacrifice” seems by definition incompatible with “maximizing fitness”?
being good to your relatives promotes your genes.
reciprocal goodness can be an advantage to both players.
doing good—precisely because it is a sacrifice—can become a signal of abundance, which makes other humans want to be my allies or mates.
people reward good and punish evil in others, because it is in their selfish interest to live among good people.
The problems caused by the evolutionary origin of goodness are also well-known: people are more likely to be good towards their neighbors who can reciprocate or towards potential sexual partners, and they are more likely to do good when they have an audience who approves of it… and less likely to do good to low-status people who can’t reciprocate, or when their activities are anonymous. (Steals money from pension funds, polutes the environment, then donates millions to a prestigious university.)
I assume that most people are “instinctively good”, that is that they kinda want to be good, but they simply follow their instincts, and don’t reflect much on them (other than rationalizing that following their instinct was good, or at least a necessary evil). Their behavior can be changed by things that affect their instincts—the archetypal example is the belief in an omniscient judging God, i.e. a powerful audience who sees all behavior, and rewards/punishes according to social norms (so now the only problem is how to make those social norms actually good). I am afraid that this ship has sailed, and that we do not really have a good replacement—any non-omniscient judge can be deceived, and any reward mechanism will be Goodharted. Another problem is that by trying to make society more tolerant and more governed by law, we also take away people’s ability to punish evil… as long as the evil takes care to only do evil acts that are technically legal, or when there is not enough legal evidence of wrongdoing.
Assuming we have a group of saints (who have the same values, and who trust each other to be saints), I am not even sure what would be the best strategy for them. Probably to cooperate with each other a lot, because there is no risk of being stabbed in the back. Try to find other saints, test them, and then admit them to the group. Notice good acts among non-saints and reward them somehow—maybe in form of lottery, when most good acts only get a “thank you”, but one in a million gets a million-dollar reward. (People overestimate their chances in lottery. This would lead them to overestimate how likely a good act is to be rewarded, which would make them do more good.) The obvious problem with rewarding good acts is that it rewards visibility; perhaps there should be a special rewards for good acts that were unlikely to get noticed. The good acts should get a social reward, i.e. telling other people about the good act and how someone was impressed.
(The sad thing is that given that we live in a clickbait society, it would not take much time until someone would publish an article about how X-ist the saints are, because the proportion of Y’s they rewarded for good deeds is not the same as the proportion of Y’s in the society. Also, this specific person rewarded for this specific good deed also happens to hold some problematic opinions, does this mean that the saints secretly support the opinion, too?)
I sometimes like to imagine a soft version of karma, like if people would be free to associate with people who are like them, then the good people would associate with other good people, the bad people would associate with other bad people, and then the bad people would suffer (because surrounded by bad people), and the good people would live nice lives (because surrounded by good people). The problem with this vision is that people are not so free to choose their neighbors (coordination is hard, moving is expensive), and also that the good people who suck at judging other people’s goodness would suffer. Not sure what is the right approach here, other than perhaps we should become a bit more judgmental, because it seems the pendulum has swung too much in the direction that you are not even allowed to criticize [an obviously horrible thing] out of concern that some culture might routinely [do the horrible thing], which would get you called out as intolerant, which is a sin much worse than [doing the horrible thing]. I’d like people to get some self-respect and say “hey, these are my values, if you disagree, fuck off”. But this of course assumes that the people who disagree actually have a place to go. Another problem is that you cannot build an archipelago, if the land is scarce, and your solution to conflicts is to walk away.
(Also, a fraction of people are literally psychopaths, so even if we devised a set of nudges to make most people behave good, it would not apply to everyone. To make someone behave good out of mere rational self-interest, they would have to believe that almost all evil deeds get detected and punished, which is very difficult to achieve.)
I usually associate things like “being evil” more with something like “part of my payoff matrix has a negative coefficient on your payoff matrix”. I.e. actively wanting to hurt people and taking inherent interest in making them worse off. Selfishness feels pretty different from being evil emotionally, at least to me.
Judgement of evil follows the same pressures as evil itself. Selfishness feels different from sadism to you, at least in part because it’s easier to find cooperative paths with selfishness. And this question really does come down to “when should I cooperate vs defect”.
If your well-being has exactly zero value in my preference function, that literally means that I would kill you in a dark alley if I believed there was zero chance of being punished, because there is a chance you might have some money that I could take. I would call that “evil”, too.
You can’t hypothesize zeros and get anywhere. MANY MANY psychopaths exist, and very few of them find it more effective to murder people for spare change than to further their ends in other ways. They may not care about you, but your atoms are useful to them in their current configuration.
There are ways of hurting people other than stabbing them, I just used a simple example.
I think there is a confusion about what exactly “selfish” means, and I blame Ayn Rand for it. The heroes in her novels are given the label “selfish” because they do not care about possibilities to actively do something good for other people unless there is also some profit for them (which is what a person with zero value for others in their preference function would do), but at the same time they avoid actively harming other people in ways that could bring them some profit (which is not what a perfectly selfish person would do).
As a result, we get quite unrealistic characters who on one hand are described as rational profit maximizers who don’t care about others (except instrumentally), but on the other hand they follow an independently reinvented deontological framework that seems like designed by someone who actually cares about other people but is in deep denial about it (i.e. Ayn Rand).
A truly selfish person (someone who truly does not care about others) would hurt others in situations where doing so is profitable (including second-order effects). A truly selfish person would not arbitrarily invent a deontological code against hurting other people, because such code is merely a rationalization invented by someone who already has an emotional reason not to hurt other people but wants to pretend that instead this is a logical conclusion derived from first principles.
Interacting with a psychopath with likely get you hurt. It will likely not get you killed, because some other way of hurting you has a better risk:benefit profile. Perhaps the most profitable way is to scam you of some money and use you to get introduced to your friends. Only once in a while a situation will arise when raping someone is sufficiently safe, or killing someone is extremely profitable, e.g. because that person stands in a way of a grand business.
I’m not sure what our disagreement actually is—I agree with your summary of Ayn Rand, I agree that there are lots of ways to hurt people without stabbing. I’m not sure you’re claiming this, but I think that failure to help is selfish too, though I’m not sure it’s comparable with active harm.
It may be that I’m reacting badly to the use of “truly selfish”—I fear a motte-and-bailey argument is coming, where we define it loosely, and then categorize actions inconsistently as “truly selfish” only in extremes, but then try to define policy to cover far more things.
I think we’re agreed that the world contains a range of motivated behaviors, from sadistic psychopaths (who have NEGATIVE nonzero terms for others’ happiness) to saints (whose utility functions weight very heavily toward other’s happiness over their own). I don’t know if we agree that “second-order effect” very often dominate the observed behaviors over most of this range. I hope we agree that almost everyone changes their behavior to some extent based on visible incentives.
I still disagree with your post that a coefficient of 0 for you in someone’s mind implies murder for pocket change. And I disagree with the implication that murder for pocket change is impossible even if the coefficient is above 0 - circumstances matter more than innate utility function.
To the OP’s point, it’s hard to know how to accomplish “make people less selfish”, but “make the environment more conducive to positive-sum choices so selfish people take cooperative actions” is quite feasible.
I believe this is exactly what it means, unless there is a chance of punishment or being hurt by victim’s self-defense or a chance of better alternative interaction with given person. Do you assume that there is always a more profitable interaction? (What if the target says “hey, I just realized that you are a psychopath, and I do not want to interact with you anymore”, and they mean it.)
Could you please list the pros and cons of deciding whether to murder a stranger who refuses to interact with you, if there is zero risk of being punished, from the perspective of a psychopath? As I see it, the “might get some pocket change” in the pro column is the only nonzero item in this model.
There always is that chance. That’s mostly our disagreement. Using real-world illustrations (murder) for motivational models (utility) really needs to acknowledge the uncertainty and variability, which the vast majority of the time “adds up to normal”. There really aren’t that many murders among strangers. And there are a fair number of people who don’t value others’ very highly.
Yes, I would make this distinction too. Yet, I submit that few people actually believe, or even say they believe, that the main problems in the world are caused by people being gratuitously or sadistically evil. There are some problems that people would explain this way: violent crime comes to mind. But I don’t think the evil hypothesis is the most common explanation given by non-rationalists for why we have, say, homelessness and poverty.
That is to say that, insofar as the common rationalist refrain of “problems are caused by incentives dammit, not evil people” refers to an actual argument people generally give, it’s probably referring to the argument that people are selfish and greedy. And in that sense, the rationalists and non-rationalists are right: it’s both the system and the actors within it.