Defecting by Accident—A Flaw Common to Analytical People

lionhearted (Sebastian Marshall)1 Dec 2010 8:25 UTC
125 points
433 comments15 min readLW linkArchive

Related to: Rationalists Should Win, Why Our Kind Can’t Cooperate, Can Humanism Match Religion’s Output?, Humans Are Not Automatically Strategic, Paul Graham’s “Why Nerds Are Unpopular”

The “Prisoner’s Dilemma” refers to a game theory problem developed in the 1950′s. Two prisoners are taken and interrogated separately. If either of them confesses and betrays the other person—“defecting”—they’ll receive a reduced sentence, and their partner will get a greater sentence. However, if both defect, then they’ll both receive higher sentences than if neither of them confessed.

This brings the prisoner to a strange problem. The best solution individually is to defect. But if both take the individually best solution, then they’ll be worst off overall. This has wide ranging implications for international relations, negotiation, politics, and many other fields.

Members of LessWrong are incredibly smart people who tend to like game theory, and debate and explore and try to understand problems like this. But, does knowing game theory actually make you more effective in real life?

I think the answer is yes, with a caveat—you need the basic social skills to implement your game theory solution. The worst-case scenario in an interrogation would be to “defect by accident”—meaning that you’d just blurt out something stupidly because you didn’t think it through before speaking. This might result in you and your partner both receiving higher sentences… a very bad situation. Game theory doesn’t take over until basic skill conditions are met, so that you could actually execute any plan you come up with.

The Purpose of This Post: I think many smart people “defect” by accident. I don’t mean in serious situations like a police investigation. I mean in casual, everyday situations, where they tweak and upset people around them by accident, due to a lack of reflection of desired outcomes.

Rationalists should win. Defecting by accident frequently results in losing. Let’s examine this phenomenon, and ideally work to improve it.

Contents Of This Post

  • I’ll define “defecting by accident.”

  • I’ll explain a common outcome of defecting by accident.

  • I’ll give some recent, mild examples of accidental defections.

  • I’ll give examples of how to turn accidental defections into cooperation.

  • I’ll give some examples of how this can make you more successful at your goals.

  • I’ll list some books I recommend if you decide to learn more on the topic.

Background—On Analytical Skills and Rhetoric

From Paul Graham’s “Why Nerds Are Unpopular” -

I know a lot of people who were nerds in school, and they all tell the same story: there is a strong correlation between being smart and being a nerd, and an even stronger inverse correlation between being a nerd and being popular. Being smart seems to make you unpopular.
[...]
The key to this mystery is to rephrase the question slightly. Why don’t smart kids make themselves popular? If they’re so smart, why don’t they figure out how popularity works and beat the system, just as they do for standardized tests?
[...]
So if intelligence in itself is not a factor in popularity, why are smart kids so consistently unpopular? The answer, I think, is that they don’t really want to be popular.
If someone had told me that at the time, I would have laughed at him. Being unpopular in school makes kids miserable, some of them so miserable that they commit suicide. Telling me that I didn’t want to be popular would have seemed like telling someone dying of thirst in a desert that he didn’t want a glass of water. Of course I wanted to be popular.
But in fact I didn’t, not enough. There was something else I wanted more: to be smart.

I believe that “defecting by accident” is a result of not learning how different phrasing of words and language can dramatically effect how well your point is taken. It’s been a general observation of mine that a lot of people in highly intellectual disciplines like mathematics, physics, robotics, engineering, and computer science/​programming look down on social skills.

Of course, they wouldn’t phrase it that way. They’d say they don’t have time for it—they don’t have time for gossip, or politics, or sugarcoating. They might say, “I’m a realist” or “I say it like it is.”

I believe this is a result of not realizing how big the difference in your effectiveness will be depending on how you phrase things, in what order, how well you appeal to another person’s emotions. People in highly analytical disciplines often care about “just the facts”—but, let’s face it, we highly analytical people are a great minority of the population.

Sooner or later, you’re going to have something you care about and you’re going to need to persuade someone who is not highly analytical. At that point, you run some serious risks of failure if you don’t understand basic social skills.

Now, most people would claim that they have basic social skills. But I’m not sure this is borne out by observation. This used to be a very key part of any educated person’s studies: rhetoric. From Wikiedpia: “Rhetoric is the art of using language to communicate effectively and persuasively. … From ancient Greece to the late 19th Century, it was a central part of Western education, filling the need to train public speakers and writers to move audiences to action with arguments.”

Rhetoric is now frequently looked down upon by highly intelligent and analytical people. Like Paul Graham says, it’s not that intellectuals can’t learn it. It’s that they think it’s not a good use of their time, that they’d rather be smart instead.

Defecting by Accident

Thus, you see highly intelligent people do what I now term “defecting by accident”—meaning, in the process of trying to have a discussion, they insult, belittle, or offend their conversational partner. They commit obvious, blatant social faux pases, not as a conscious decision of the tradeoffs, but by accident because they don’t know better.

Sometimes defecting is the right course of action. Sometimes you need to break from whoever you’re negotiating with, insist that things are done your way, even at their expense, and take the consequences that may arise from that.

But it’s rarely something you should do by accident.

I’ll give specific, clear examples in a moment, but before I do so, let’s look at a general example of how this can happen.

If you’re at a meeting and someone gives a presentation and asks if anyone has questions, and you ask point-blank, “But we don’t have the budget or skills to do that, how would we overcome that?”—then, that seems like a highly reasonable question. It’s probably very intelligent.

What normal people would consider, though, is how this affects the perception of everyone in the room. To put it bluntly—it makes the presenter look very bad.

That’s okay, if you decide that that’s an acceptable part of what you’re doing. But you now have someone who is likely to actively work to undermine you going forwards. A minor enemy. Just because you asked a question casually without thinking about it.

Interestingly, there’s about a thousand ways you could be diplomatic and tactful to address the key issue you have—budgeting/​staffing—without embarrassing the presenter. You could take them aside quietly later and express your concern. You could phrase it as, “This seems like an amazing idea and a great presentation. I wonder how we could secure the budgeting and get the team for it, because it seems like it’d be a profitable if we do, and it’d be a shame to miss this opportunity.”

Just by phrasing it that way, you make the presenter look good even if the option can’t be funded or staffed. Instead of expressing your concern as a hole in their presentation, you express it as a challenge to be overcome by everyone in the room. Instead of your underlying point coming across as “your idea is unfeasible,” it comes across as, “You’ve brought this good idea to us, and I hope we’re smart enough to make it work.”

If the real goal is just to make sure budgeting and funding is taken care of, there’s many ways to do that without embarrassing and making an enemy out of the presenter.

Defecting by accident is lacking the awareness, tact, and skill to realize what the secondary effects of your actions are and act accordingly to win.

This is a relatively basic problem that the majority of “normal” people understand, at least on a subconscious level. Most people realize that you can’t just show up a presenter and make them look bad. Or at least, you should expect them to be hostile to you if you do. But many intelligent people say, “What the hell is his problem? I just asked a question.”

This is due to a lack of understanding of social skills, diplomacy, tact, and yes, perhaps “politics”—which are unfortunately a reality of the world. And again, rationalists should win. If your actions are leading to hostility and defection against you, then you need to consider if your actions are the best possible.

“Why Our Kind Can’t Cooperate”

Eliezer’s “Why Our Kind Can’t Cooperate” is a masterpiece. I’m only going to excerpt three parts, but I’d recommend the whole article.

From when I was still forced to attend, I remember our synagogue’s annual fundraising appeal. It was a simple enough format, if I recall correctly. The rabbi and the treasurer talked about the shul’s expenses and how vital this annual fundraise was, and then the synagogue’s members called out their pledges from their seats.

Straightforward, yes?
Let me tell you about a different annual fundraising appeal. One that I ran, in fact; during the early years of a nonprofit organization that may not be named. One difference was that the appeal was conducted over the Internet. And another difference was that the audience was largely drawn from the atheist/​libertarian/​technophile/​sf-fan/​early-adopter/​programmer/​etc crowd. (To point in the rough direction of an empirical cluster in personspace. If you understood the phrase “empirical cluster in personspace” then you know who I’m talking about.)
I crafted the fundraising appeal with care. By my nature I’m too proud to ask other people for help; but I’ve gotten over around 60% of that reluctance over the years. The nonprofit needed money and was growing too slowly, so I put some force and poetry into that year’s annual appeal. I sent it out to several mailing lists that covered most of our potential support base.
And almost immediately, people started posting to the mailing lists about why they weren’t going to donate. Some of them raised basic questions about the nonprofit’s philosophy and mission. Others talked about their brilliant ideas for all the other sources that the nonprofit could get funding from, instead of them. (They didn’t volunteer to contact any of those sources themselves, they just had ideas for how we could do it.)
Now you might say, “Well, maybe your mission and philosophy did have basic problems—you wouldn’t want tocensor that discussion, would you?”
Hold on to that thought.
Because people were donating. We started getting donations right away, via Paypal. We even got congratulatory notes saying how the appeal had finally gotten them to start moving. A donation of $111.11 was accompanied by a message saying, “I decided to give **** a little bit more. One more hundred, one more ten, one more single, one more dime, and one more penny. All may not be for one, but this one is trying to be for all.”
But none of those donors posted their agreement to the mailing list. Not one.

So far as any of those donors knew, they were alone. And when they tuned in the next day, they discovered not thanks, but arguments for why they shouldn’t have donated. The criticisms, the justifications for not donating—only those were displayed proudly in the open.
As though the treasurer had finished his annual appeal, and everyone not making a pledge had proudly stood up to call out justifications for refusing; while those making pledges whispered them quietly, so that no one could hear.

Indeed, that’s a problem. Eliezer continues:

“It is dangerous to be half a rationalist.”

And finally, this point, which is magnificent -

Our culture puts all the emphasis on heroic disagreement and heroic defiance, and none on heroic agreement or heroic group consensus. We signal our superior intelligence and our membership in the nonconformist community by inventing clever objections to others’ arguments. Perhaps that is why the atheist/​libertarian/​technophile/​sf-fan/​Silicon-Valley/​programmer/​early-adopter crowd stays marginalized, losing battles with less nonconformist factions in larger society. No, we’re not losing because we’re so superior, we’re losing because our exclusively individualist traditions sabotage our ability to cooperate.

On Being Pedantic, Sarcastic, Disagreeable, Non-Complimentary, and Otherwise Defecting by Accident

You might not realize it, but in almost all of human civilization it’s considered insulting to just point out something wrong someone is doing without any preface, softening, or making it clear why you’re doing it.

It’s taken for granted in some blunt, “say it like it is” communities, but it’s usually taken as a personal attack and a sign of animosity in, oh, 90%+ of the rest of civilization.

In these so-called “normal people’s societies,” correcting them in front of their peers will be perceived as trying to lower them and make them look stupid. Thus, they’ll likely want to retaliate against you, or at least not cooperate with you.

Now, there’s a time and place to do this anyways. Sometimes there’s an emergency, and you don’t have time to take care of people’s feelings, and just need to get something done. But surfing the internet is not that time.

I’m going to take some example replies from a recent post I made to illustrate this. There’s always a risk in doing this of not being objective, but I think it’s worth it because (1) I tend to read every reply to me and carefully reflect on it for a moment, (2) I understand exactly my first reactions to these comments, and (3) I won’t have to rehash criticisms of another person. Take a grain of salt with you since I’m looking at replies to myself originally, but I think I can give you some good examples.

The first thing I want to do is take a second to mention that almost everyone in the entire world gets emotionally invested in things they create, and are also a little insecure about their creations. It’s extraordinarily rare that people don’t care what others’ think of their writing, science, or art.

Criticism has good and bad points. Great critics are rare, but they actually make works of creation even in critique. A great critic can give background, context, and highlight a number of relevant mainstream and obscure works through history that the piece they’re critiquing reminds them of.

Good critique is an art of creation in and of itself. But bad critique—just blind “that’s wrong” without explaining why—tends to be construed as a hostile action and not accomplish much, other than signalling that “heroic disagreement” that Eliezer talks about.

I recently wrote a post titled, “Nahh, that wouldn’t work”. I thought about it for around a week, then it took me about two hours to think it through, draw up key examples on paper, choose the most suitable, edit, and post it. It was generally well-received here on LW and on my blog.

I’ll show you three comments on there, and how I believe they could be subtly tweaked.

1.

> I wizened up,
I don’t think that’s the word you want to use, unless you’re talking about how you finally lost those 20 pounds by not drinking anymore.

2.

FWIW, I think posts like this are more valuable the more they include real-world examples; it’s kind of odd to read a post which says I had theory A of the world but now I hold theory B, without reading about the actual observations. It would be like reading a history of quantum mechanics or relativity with all mentions of things like the laser or double-slit experiment or Edding or Michelson-Morley removed.

3.

An interesting start, but I would rather see this in Discussion—it’s not fully adapted yet, I think...

Now, I spend a lot of time around analytical people, so I take no offense at this. But I believe these are good examples of what I’d call “accidental defection”—this is the kind of thing that produces a negative reaction in the person you’re talking to, perhaps without you even noticing.

#1 is kind of clever pointing out a spelling error. But you have to realize, in normal society that’s going to upset and make hostile the person you’re addressing. Whether you mean to or not, it comes across as, “I’m demonstrating that I’m more clever than you.”

There’s a few ways it could be done differently. For instance, an email that says, “Hey Sebastian, I wanted to give you a heads up. I saw your recent post, but you spelled “wisen” as “wizen”—easy spelling error to make, since they’re uncommonly used words, but I thought you should know. “Wizen” means for things to dry up and lose water. Cheers and best wishes.”

That would point out the error (if that’s the main goal), and also engender a feeling of gratitude in whoever received it (me, in this case). Then I would have written back, “Hey, thanks… I don’t worry about spelling too much, but yeah that one’s embarrassing, I’ll fix it. Much appreciated. Anyways, what are you working on? How can I help?”

I know that’s how I’d have written back, because that’s how I generally write back to someone who tries to help me out. Mutual goodwill, it’s a virtuous cycle.

Just pointing out someone is wrong in a clever way usually engenders bad will and makes them dislike you. The thing is, I know that’s not the intention of anyone here—hence, “defecting by accident.” Analytical people often don’t even realize they’re showing someone up when they do it.

I’m not particularly bothered. I get the intent behind it. But normal people are going to be ultra-hostile if you do it to them. There’s other ways, if you feel the need to point it out publicly. You could “soften” it by praising first—“Hey, some interesting points in this one… I’ve thought about a similar bias of not considering outcomes if I don’t like what it’d mean by the world. By the way, you probably didn’t mean wizen there...”—or even just saying, “I think you meant ‘wisen’ instead of ‘wizen’”—with links to the dictionary, maybe. Any of those would go over better with the original author/​presenter whom you’re pointing out the error to.

Let’s look at point #2. “FWIW, I think posts like this are more valuable the more they include real-world examples; it’s kind of odd to read a post which says I had theory A of the world but now I hold theory B, without reading about the actual observations.”

This is something which makes people trying to help or create shake their head. See, it’s potentially a good point. But after someone takes some time to create something and give it away for free, then hearing, “Your work would be more valuable if you did (xyz) instead. Your way is kind of odd.”

People generally don’t like that.

Again, it’s trivially easy to write that differently. Something like, “Thanks for the post. I was wondering, you mentioned (claim X), but I wonder if you have any examples of claim X so I can understand it better?”

That one has—gratitude, no unnecessary criticism, explains your motivation. All of which are good social skill points, especially the last one as written about in Cialdini’s “Influence”—give a reason why.

#3 - “An interesting start, but I would rather see this in Discussion—it’s not fully adapted yet, I think...”

Okay. Why?

The difference between complaining and constructive work is looking for solutions. So, “There’s some good stuff in here, but I think we could adapt it more. One thing I was thinking is (main point).”

Becoming More Self-Aware and Strategic; Some Practical Social Guidelines

From Anna Salamon’s “Humans Are Not Automatically Strategic” -

But there are clearly also heuristics that would be useful to goal-achievement (or that would be part of what it means to “have goals” at all) that we do not automatically carry out. We do not automatically:

  • (a) Ask ourselves what we’re trying to achieve;

  • (b) Ask ourselves how we could tell if we achieved it (“what does it look like to be a good comedian?”) and how we can track progress;

  • (c) Find ourselves strongly, intrinsically curious about information that would help us achieve our goal;

Anna points out that people don’t automatically ask what they’re trying to achieve. You don’t, necessarily, ask what you’re trying to achieve.

But I would recommend you do ask that before speaking up socially. At least for a while, until you’ve got the general patterns figured out.

If you don’t, you run the risk of antagonizing and making people hostile to you who would otherwise cooperate and work with you.

Now, I’ve heard smart people say, “I don’t have time for that.” This is akin to saying, “I don’t have time to achieve what I want to achieve.”

Because it doesn’t take much time, and it makes you much more effective. Asking, “What am I trying to achieve here?” goes a long way.

When commenting on a discussion site, who are you writing for? For the author? For the regular readers? What’s your point in replying? If your main point is just to “get to truth and understanding,” then what should your secondary considerations be? If there’s a conflict between the two, would you prefer to encourage the author to write more, or to look clever by pointing out a pedantic point?

I understand where you’re coming from, because I used to come from the same place. I was the kid who argued with teachers when they were wrong, not realizing the long term ramifications of that. People matter, and people’s feelings matter, especially if they have sway over your life, but even if they don’t have sway over your life.

To that, here’s some suggestions I think would make you more effective:

  • Generally, be gracious and thankful. This goes immensely far. Things like starting a reply with, “Thanks for this” or “Thanks for sharing these insights.”

  • Praising someone makes it more likely they’ll accept your criticisms. “I thought your point A was excellent, however point B...”

  • If you’re going to disagree, summarize the person’s main argument beforehand—this has a few positive effects. First, it forces you make sure you actually understand. Second, if the author has a different main point and wasn’t clear, that comes out. Third, it shows some respect that you actually took the time to read and understand the post. So you could write, “I know your main argument is A, but I wanted to explore your minor point X.”

  • If you think something is wrong, give an explanation of what would be correct and better. “I enjoyed this post a lot—thanks for that—but one thing that’s tough for me is that all the examples are about martial arts, and I don’t really understand martial arts so much. Maybe next time you could provide some examples from other fields? For instance, I remember reading you’re an accountant and you write poetry, maybe some examples from there?”

  • If you point out something is wrong, do your best to make the mistake-maker not feel stupid. This makes them massively appreciate that. “Hey, you got your math on example X wrong… I think it actually works to 11.7. Anyways, I only recognize that because I made that mistake dozens of times myself, it’s a common one to make, just wanted to point it out.”

  • Explain why you care about a point. This has a few positive effects. First, it lets the author cater a reply to exactly what you want. Second, you’d be amazed at how many people assume evil intent and worst-possible motives—it neutralizes that. Third, it forces you to think through how you’d like things to be, which is again good. “Hey man, I really liked this post, but I wonder if you could have split it into pieces and made it a three-parter? I ask because I surf the web from work, and I can only read in 10 minute chunks… longer posts are harder for me to get through, and I like reading your writing.”

  • Consider correcting someone privately while praising them publicly. This combination has been observed to engender loyalty and good feelings throughout history. I recently read an example of a samurai encouraging lords to do this from the early 1700′s book “Hagakure.” It works.

  • Consider dropping it altogether if it’s not a big deal. This about learning to prioritize—I had someone comment on my site thinking mistakenly that The Richest Man in Babylon and The Greatest Salesman in the World were by the same author. It wasn’t, but who cares? It makes no difference. It’s not worth pointing it out—almost everyone has an aversion to being corrected, so only do it if there’s actually tangible gain. Otherwise, go do something more important and not engender the potential bad will.

Following some of these simple points will make you much more effective socially. I feel like a lot of times analytical and intelligent people study really hard, difficult problems, while ignoring basic considerations that have much more immediate and larger impact.

Further reading:

Edit: Lots of comments on this. 130 and counting. The most common criticism seems to be that adding fluff is a waste of time, insincere, and reduces signal:noise ratio. I’d encourage you to actually try it instead of just guessing—a quick word of thanks or encouragement before criticizing creates a more friendly, cooperative environment and works well. It doesn’t take very long, and it doesn’t detract from S:N ratio much, if at all.

Don’t just guess here. Try it out for a month. I think you’ll be amazed at how differently people react to you, and the uptake on your suggestions and feedback and ability to convince and teach people. Of course, you can construct examples of going overboard and it being silly. But that’s not required—just try to make everything 10% more gracious, and watch how much your effectiveness increases.

lionhearted (Sebastian Marshall)1 Dec 2010 8:25 UTC
125 points
433 comments15 min readLW linkArchive