Unconscious Competence and Counter-Incentives

[Some things are better done when you don’t think about them. This has an interesting extension when you consider situations where the default response does very little to actually solve things. Maybe we’ve been Goodharted by social incentives.]

Cue the four stages of competence.

The four stages represent a framework for modeling how people get better at a task, from unconscious to conscious incompetence, followed by conscious to unconscious competence.

In other words:

Unconscious incompetence: :“What are these two sticks for?”
Conscious incompetence: “Man, I really suck at using these chopsticks.”
Conscious competence: “Left…right…squeeze…there we go! Grabbed that cucumber slice!”
Unconscious competence: “Nomnomnom…what chopsticks?”

What gets touted as the highest stage of mastery here is that of unconscious competence, when you get so good at a task that you don’t need to focus on the task at hand to do well at it.

Some quick examples that come to mind:

Magicians are not focused on the coins hidden in their hand when they perform.
Teachers are not reviewing the content material in their heads when they teach.
Athletes are not focused on the step-by-step process of moving their feet when they run.

One common thread to observe about the above three examples is that they all involve a “performance” of sorts. Especially in the cases of physical activities, it’s easy to chalk it up to “muscle memory” and consider the case of unconscious competence closed. What is interesting for me right now, though, is the paradoxical nature of the generalized concept where thinking about how to do X well is antithetical to doing X well.

A few more examples:

Thinking about how you are going to die is not conducive to a state of mind where you can take meaningful action to change that fact.
If you’re socially anxious, focusing on how you’re talking to someone will not make it easier to hold a good conversation.
(UNSONG spoilers!) Wanting to go to Hell for the altruistic reason of saving everyone suffering there makes it impossible to get there in the first place.

There’s been a lot of talk recently on LessWrong about fake frameworks and how they relate to what’s true and/or correct. I think the above three examples do a good job of showcasing the value of framings other than the directly-optimizing/overanalyzing mindset rationalists are often stereotypically depicted as having.

And if you really were trying to do well, then the right thing to do, under a direct optimizing mindset, would be to self-modify such that you weren’t thinking that much about how well you were doing. (In order to do well.)

The big picture takeaway here is to just be mindful of how your internal state can have instrumental effects on what you do—including the states that are typically induced when you’re trying to do something.

Another interesting phenomenon that seems like a similar instance of this property is when you generalize it to the ways people respond to problems. In particular, certain systems seem to be set up in such a way that selection pressures mean that the people you want to affect are the ones least likely to respond to whatever intervention you’ve set up.

Once again, to triangulate this concept, here are three examples:

Telling someone who is angry to “calm down” doesn’t tend to calm them down because people who are angry don’t respond well to reasoned claims. (Ditto for saying things like “cheer up” and “don’t worry” to people who could benefit most from what those platitudes refer to.)
Yelling at people who you think are being uncharitable on the other side of whatever debate you’re on isn’t going to work because the people on the other side most likely to listen aren’t the people you’re trying to reach with your voice.
A traffic cop whistling at jaywalking pedestrians doesn’t do much because someone who didn’t care about the walk signal in the first place also isn’t likely to respond to the cop.

What seems striking here to me is how quickly these default responses spring to mind in each of the systems. See a sad friend? You tell them that “it’ll be okay”. Feel outraged by the other side? An outburst jumps straight to your lips.

These responses are downright ineffective. They don’t work, and we should know from doing them time and time again, that they don’t work. So why do these responses persist?

It’s things like this which make me sympathetic to a Hansonian view of human behavior. In these situations, I don’t think we’re taking a good look at the results of our actions. Instead, it’s like we’re just going through the motions to send a message about the roles we play:

Going off our above examples:

“Calm down.” (“I’m a sensible human, and this is what sensible people say. By saying this, I can show how I’m taking the ‘rational’ high ground in this discussion.”)
<Insert outburst> (“You said something I disagreed with, and now I’m showing you how angry that made me. I’m warning you, don’t say more things which will provoke me!”)
<Whistles> (“Hey, you’re breaking social norms! As someone who represents norms, I need to make your defection clear for everyone else to see, and to reinforce my representation of norms!”)

I feel like this is one the areas in life where we’ve Goodharted ourselves. It’s not as if people don’t care about the results of the actions they take. People do care about what happens to them [citation needed]. Rather, somewhere along the line, we’ve convinced ourselves that the effective action is the one which just so happens to do a very good job of serving a social function but quite a sucky job of doing the actual thing.