What you can’t say to a sympathetic ear
Suppose we live in a society where it is strongly frowned upon to believe that an onion is a fruit. It is ok to disagree about what defines ‘fruit’, or what Allium varieties are onions. But none of this will get you off the hook—you had just better not suggest that an onions is a fruit.
You don’t think about the issue much yourself. If you had to, you would probably agree with the consensus view that the onion is not a fruit, given a few clarifications of the question. If you were allowed, you would probably admit that you don’t care much about the question. And that you would kind of prefer that it was possible to discuss the issue calmly and without accusations of transcendent evil.
However none of these things is relevant in the real world, because you daren’t even advocate for calmer consideration of the onion classification issue. People would infer that (in a sense) you don’t want to punish people who say that an onion is a fruit. And (in a different sense) not punishing it is much like endorsing it. Punishing non-punishers is an important part of cooperation.
Now suppose you and I are chatting over lunch in a work cafeteria, and I glance furtively around and then lean over to you with gleaming eyes, and whisper that I am making fruit soup tonight, and ahem, there are many people who would cry if they watched me cutting up the fruit for it.
You see what I’m saying. Nobody else seems to have heard. Are you annoyed? Do you think worse of me?
My guess is yes, at least quite plausibly.
And you are not annoyed because you find my comments troubling in their own right. You disagree with them, but don’t find them intrinsically offensive. Outside the context of our society, you wouldn’t mind.
You might be offended that I am willing to suggest an onion is a fruit in your presence in spite of knowing that most people would be unhappy about this. But suppose that we know each other well, and you know I know you are hard to hurt, even with grievous categorization errors.
I think you still have a strong reason to be annoyed. Which is that I am intentionally taking an action that the rest of the world thinks you are strongly obliged to punish—for instance, by threatening to stop associating with me unless I have an amazing excuse for what kind of seizure took over my mouth. Which means you must decide on the spot whether to punish me (at a cost to our relationship) or implicitly collude a bit with my renegade controversial-thing-saying faction. At a cost to your relationship with the world, because if they learned of this, they would hate you.
This makes my implied classification of onions as fruit into an ultimatum: ‘Me or the rest of society?’ If it is intentional, then it is a test of our friendship, at your expense. It’s like randomly saying ‘ok, if you really care about our friendship then steal $10 from your grandmother to prove it’.
Saying that onions are fruit quietly to you is holding our friendship hostage unless you shift your alliances away from the rest of the world and toward me. Or, more likely, it is an accident that still puts you in this position.
And it is very annoying to have your valuables taken hostage, and even more annoying to be threatened on short notice, with a deadline, so that you can’t just put it at the bottom of your to-do list and deal with it another time.
I hadn’t explicitly noticed that this kind of dynamic existed before (and it may not), but I think it might play a large part in my own feelings, on both sides of situations that are a bit like this.
I am sometimes annoyed when people reveal disagreeable views to me, even if I don’t especially disagree with them. Which is a bit surprising, on the face of it. And other times, I find myself in the position of wanting to say things that may sound controversial, and feeling hesitant, in part for the other person’s sake. So I got to thinking about the possible ways that could harm someone. And imagining myself in their shoes, this is the kind of harm I expected. I have not much idea if others feel the same way in these circumstances, or would construe the situation similarly in terms of game theory.
This might all seem pretty unimportant, being as it is a speculative and hand-wavey analysis of an already obscure social situation. But the existence of multiple reasons to be offended by officially offensive statements—even if you are sympathetic to them—means that social bans on views should be more stable than you might think. It’s one of those things where even if everyone comes to privately believe that onions are indeed fruit, and also thinks that nobody should be punished for saying this, and they can all talk to each other, everyone might still end up saying that onions aren’t fruit forever.
This means sanctions on speech aren’t just costly because they make it hard for individuals to hear ideas that might turn out to be true. They are more costly than that, because even if every individual manages to hear the ideas, and they are good, still they might not be able to update their behavior or the social consensus. And if everyone has to talk and behave as if a claim is false, we have lost a lot of the value of knowing that it is true.
To successfully condemn a view socially is to lock that view in place with a coordination problem. We could all freely identify onions as we wanted, but we have to all at once decide to change the norms, or else we get punished. And changing the norms would be hard to arrange at the best of times, but is harder when trying to arrange it warrants punishment.
If this analysis is correct, I think this situation should raise the bar for condemning views, because it makes it even harder for future people to undo our mistakes where we have erred. Condemnation is more permanent.
ETA Sept 3 2017: I am on reflection happy for people to tell me their controversial views if they are interesting—I bring up my slight feeling of annoyance about it as evidence that it is imposing some cost. But I am usually willing to pay the cost.
I think this remains an outstanding, top-tier problem in group rationality. I feel like I encounter it constantly. I’m surprised this post doesn’t have more engagement.