Your Price for Joining

Eliezer Yudkowsky26 Mar 2009 7:16 UTC
87 points
58 comments4 min readLW linkArchive

In the Ultimatum Game, the first player chooses how to split $10 between themselves and the second player, and the second player decides whether to accept the split or reject it—in the latter case, both parties get nothing. So far as conventional causal decision theory goes (two-box on Newcomb’s Problem, defect in Prisoner’s Dilemma), the second player should prefer any non-zero amount to nothing. But if the first player expects this behavior—accept any non-zero offer—then they have no motive to offer more than a penny. As I assume you all know by now, I am no fan of conventional causal decision theory. Those of us who remain interested in cooperating on the Prisoner’s Dilemma, either because it’s iterated, or because we have a term in our utility function for fairness, or because we use an unconventional decision theory, may also not accept an offer of one penny.

And in fact, most Ultimatum “deciders” offer an even split; and most Ultimatum “accepters” reject any offer less than 20%. A 100 USD game played in Indonesia (average per capita income at the time: 670 USD) showed offers of 30 USD being turned down, although this equates to two week’s wages. We can probably also assume that the players in Indonesia were not thinking about the academic debate over Newcomblike problems—this is just the way people feel about Ultimatum Games, even ones played for real money.

There’s an analogue of the Ultimatum Game in group coordination. (Has it been studied? I’d hope so...) Let’s say there’s a common project—in fact, let’s say that it’s an altruistic common project, aimed at helping mugging victims in Canada, or something. If you join this group project, you’ll get more done than you could on your own, relative to your utility function. So, obviously, you should join.

But wait! The anti-mugging project keeps their funds invested in a money market fund! That’s ridiculous; it won’t earn even as much interest as US Treasuries, let alone a dividend-paying index fund.

Clearly, this project is run by morons, and you shouldn’t join until they change their malinvesting ways.

Now you might realize—if you stopped to think about it—that all things considered, you would still do better by working with the common anti-mugging project, than striking out on your own to fight crime. But then—you might perhaps also realize—if you too easily assent to joining the group, why, what motive would they have to change their malinvesting ways?

Well… Okay, look. Possibly because we’re out of the ancestral environment where everyone knows everyone else… and possibly because the nonconformist crowd tries to repudiate normal group-cohering forces like conformity and leader-worship...

...It seems to me that people in the atheist/​libertarian/​technophile/​sf-fan/​etcetera cluster often set their joining prices way way way too high. Like a 50-way split Ultimatum game, where every one of 50 players demands at least 20% of the money.

If you think how often situations like this would have arisen in the ancestral environment, then it’s almost certainly a matter of evolutionary psychology. System 1 emotions, not System 2 calculation. Our intuitions for when to join groups, versus when to hold out for more concessions to our own preferred way of doing things, would have been honed for hunter-gatherer environments of, e.g., 40 people all of whom you knew personally.

And if the group is made up of 1000 people? Then your hunter-gatherer instincts will underestimate the inertia of a group so large, and demand an unrealistically high price (in strategic shifts) for you to join. There’s a limited amount of organizational effort, and a limited number of degrees of freedom, that can go into doing things any one’s person way.

And if the strategy is large and complex, the sort of thing that takes e.g. ten people doing paperwork for a week, rather than being hammered out over a half-hour of negotiation around a campfire? Then your hunter-gatherer instincts will underestimate the inertia of the group, relative to your own demands.

And if you live in a wider world than a single hunter-gatherer tribe, so that you only see the one group representative who negotiates with you, and not the hundred other negotiations that have taken place already? Then your instincts will tell you that it is just one person, a stranger at that, and the two of you are equals; whatever ideas they bring to the table are equal with whatever ideas you bring to the table, and the meeting point ought to be about even.

And if you suffer from any weakness of will or akrasia, or if you are influenced by motives other than those you would admit to yourself that you are influenced by, then any group-altruistic project which does not offer you the rewards of status and control, may perhaps find itself underserved by your attentions.

Now I do admit that I speak here primarily from the perspective of someone who goes around trying to herd cats; and not from the other side as someone who spends most of their time withholding their energies in order to blackmail those damned morons already on the project. Perhaps I am a little prejudiced.

But it seems to me that a reasonable rule of thumb might be as follows:

If, on the whole, joining your efforts to a group project would still have a net positive effect according to your utility function—

(or a larger positive effect than any other marginal use to which you could otherwise put those resources, although this latter mode of thinking seems little-used and humanly-unrealistic, for reasons I may post about some other time)

—and the awful horrible annoying issue is not so important that you personally will get involved deeply enough to put in however many hours, weeks, or years may be required to get it fixed up—

—then the issue is not worth you withholding your energies from the project; either instinctively until you see that people are paying attention to you and respecting you, or by conscious intent to blackmail the group into getting it done.

And if the issue is worth that much to you… then by all means, join the group and do whatever it takes to get things fixed up.

Now, if the existing contributors refuse to let you do this, and a reasonable third party would be expected to conclude that you were competent enough to do it, and there is no one else whose ox is being gored thereby, then, perhaps, we have a problem on our hands. And it may be time for a little blackmail, if the resources you can conditionally commit are large enough to get their attention.

Is this rule a little extreme? Oh, maybe. There should be a motive for the decision-making mechanism of a project to be responsible to its supporters; unconditional support would create its own problems.

But usually… I observe that people underestimate the costs of what they ask for, or perhaps just act on instinct, and set their prices way way way too high. If the nonconformist crowd ever wants to get anything done together, we need to move in the direction of joining groups and staying there at least a little more easily. Even in the face of annoyances and imperfections! Even in the face of unresponsiveness to our own better ideas!

In the age of the Internet and in the company of nonconformists, it does get a little tiring reading the 451st public email from someone saying that the Common Project isn’t worth their resources until the website has a sans-serif font.

Of course this often isn’t really about fonts. It may be about laziness, akrasia, or hidden rejections. But in terms of group norms… in terms of what sort of public statements we respect, and which excuses we publicly scorn… we probably do want to encourage a group norm of:

If the issue isn’t worth your personally fixing by however much effort it takes, and it doesn’t arise from outright bad faith, it’s not worth refusing to contribute your efforts to a cause you deem worthwhile.

Eliezer Yudkowsky26 Mar 2009 7:16 UTC
87 points
58 comments4 min readLW linkArchive