Recent discussion on naming concepts mostly focuses on arguments in favor, noting only a few caveats, as LW user Conor Moreton in Why and How to Name Things:
What you lose by the proliferation of jargon is ease-of-entry and cross-cultural intelligibility and hard-drive space in the brains of people trying to track all of it.
I think you lose more than that, so I’ll try to name [sic] a few more reasons your caution might run deeper.
1. The nomothetic fallacy
Diagnosis isn’t a cure, but it can bring a sense of relief untethered from prognosis. You might still have the same symptoms that still need the same treatment, but it feels like a large part of the problem has been solved. This is, I’d argue, a decent chunk of why knowing about biases can hurt you. Don’t get complacent just because you have a name for something.
2. Weaponized rationality
Another big chunk of “knowing about biases can hurt you” comes from wielding concepts not in introspection but against others. (Like many of the considerations below, taking this as compelling is a general argument against much of the rationality project. But proliferating jargon, to my eye, makes most of the problems here relatively worse for external use as compared to internal use.)
3. Being wrong
Sometimes your analysis of the thing you’re trying to crystallize just isn’t very good. Your analogy doesn’t play out the way you think, the pattern doesn’t actually exist, you’re not carving reality at the joints, your framing could be better, it’s an inappropriate level of abstraction anywhere you’d actually want to use it. But, hey, the name stuck, so you can communicate all that in just a word!
4. Lossy compression
Maybe you got it right this time, but the name doesn’t capture everything. In practice, nobody’s going to remember your entire blog post every time someone utters the title. I hope you didn’t need that nuance. But at least this one went viral!
5. Thinking on the page
From Conor’s post:
When you define a concept rigorously and clearly, you almost always learn new things from playing around with their edges and trying to get them to interface with other rigorous, clear concepts.
And if you don’t define it rigorously and clearly—as is the case for pretty much everything happening here—if you play directly with the compressed concepts and string them together that way, oops, you just proved that 2+2=5. You lost track of the actual stuff the name referred to, the rules for manipulating it while preserving truth. You thought you were post-rigor when there isn’t even a rigorous stage. You were thinking on the page. [Oh, look, links to named chunked concepts. What are they doing here, of all places?]
Combining frameworks and making analogies between your concepts is a good way of generating new ideas, but it’s far from rigorous and if anything encourages sloppiness if you’re not careful to recognize that it’s less formal reasoning and more another way of pointing to where the new things to learn might be.
6. Illusion of transparency
People will assume they know what you mean. You’ll assume you know what they mean in reply—that is, what a coincidence, that they know what you mean. They don’t.
7. Lifting the pot by one handle
The trouble with rationalist skills is that the opposite of every rationalist skill is also a rationalist skill.
We have the Inside View, and the Outside View. Overconfidence is a problem, but so is underconfidence. You’re supposed to listen to the tiniest note of mental discord, yet sometimes it’s necessary to shut loud mental voices out. And while knowing the standard catalog of biases is obviously crucial for the aspiring rationalist, it can also hurt you. Et cetera, et cetera.
Is it lotus eating or self-care? Should you reverse any advice you hear or not? Remembering a blogpost title isn’t going to tell you which situation you’re in; it’s at best a handle for noticing you’re in a situation where you might want to course correct following deeper analysis. But implicit in the assignment of handles is a claim that you ought to adjust more than you do in a certain direction. And you don’t want achieving a balance of opposite mistakes to depend on the relative catchiness of named concepts. When you lift the pot by one handle, the soup spills out.
(This isn’t, on its own, an argument against giving your ideas catchy names, so much as an argument for spending plenty of time discussing trade-offs and how to do the necessary deeper analysis. But recall that it’s still mostly the handle that gets remembered.)
8. Reification
You’ll tend to treat your named ideas as more concrete than they are, as meaning one particular thing to everyone, as having agency or causal powers, as unchanging and non-reframeable, as universal and context-independent, as territory, as binary, as objective.
(I wonder if this is related to a lot of our apparent confusion about double crux—people (including me) questioning its status as The Technique, while Duncan keeps trying to clarify how it’s not about executing an algorithm, it’s about internalizing what generates the algorithm; it’s not designed for use in earnest, it’s a pedagogical tool for practicing mental habits; it’s not about Double Crux, it’s about double-cruxiness.)
9. Rounding error
Conor again:
[What you gain by the proliferation of jargon includes] being embedded in a culture where people are diligently seeking out and popularizing such distinctions makes a given individual far more likely to pay attention to subtle distinctions themselves, accelerating the process of cultural accumulation of nuance and detail.
Or, well, the opposite of that, where you round everything off to ingroup-approved jargon.
***
You can’t stop titling your blog posts, but don’t force it, or you’ll end up with virality tracking the wrong things (more so than usual, anyway). In particular, maybe don’t optimize so much towards things like catchiness, metaphorical weight, uniqueness, and communicability—even for the sake of rapid, rigorous, distinction-rich, high-level discussion (or perhaps especially for that sake, since this kind of chunking is either unnecessary or destructive for all of these but speed)—over choosing words that prepare people to read and understand what you say.
(And even as far as speed of communication, what’s weird is that the rationalist writing stereotype is absurdly prolix—it feels like the compression isn’t happening where it matters, just where it feels powerful for hiding complex abstraction in a minimally intelligible way.)
You shouldn’t stop trying to draw more distinctions, but maybe you can do that without encouraging one particular way of pointing to your distinction to reify and metastasize.
You won’t stop sloppily compressing and combining complex ideas, but maybe worry more about unpacking things, giving examples, checking understanding, and asking questions.
Against naming things, and so on
Recent discussion on naming concepts mostly focuses on arguments in favor, noting only a few caveats, as LW user Conor Moreton in Why and How to Name Things:
I think you lose more than that, so I’ll try to name [sic] a few more reasons your caution might run deeper.
1. The nomothetic fallacy
Diagnosis isn’t a cure, but it can bring a sense of relief untethered from prognosis. You might still have the same symptoms that still need the same treatment, but it feels like a large part of the problem has been solved. This is, I’d argue, a decent chunk of why knowing about biases can hurt you. Don’t get complacent just because you have a name for something.
2. Weaponized rationality
Another big chunk of “knowing about biases can hurt you” comes from wielding concepts not in introspection but against others. (Like many of the considerations below, taking this as compelling is a general argument against much of the rationality project. But proliferating jargon, to my eye, makes most of the problems here relatively worse for external use as compared to internal use.)
3. Being wrong
Sometimes your analysis of the thing you’re trying to crystallize just isn’t very good. Your analogy doesn’t play out the way you think, the pattern doesn’t actually exist, you’re not carving reality at the joints, your framing could be better, it’s an inappropriate level of abstraction anywhere you’d actually want to use it. But, hey, the name stuck, so you can communicate all that in just a word!
4. Lossy compression
Maybe you got it right this time, but the name doesn’t capture everything. In practice, nobody’s going to remember your entire blog post every time someone utters the title. I hope you didn’t need that nuance. But at least this one went viral!
5. Thinking on the page
From Conor’s post:
And if you don’t define it rigorously and clearly—as is the case for pretty much everything happening here—if you play directly with the compressed concepts and string them together that way, oops, you just proved that 2+2=5. You lost track of the actual stuff the name referred to, the rules for manipulating it while preserving truth. You thought you were post-rigor when there isn’t even a rigorous stage. You were thinking on the page. [Oh, look, links to named chunked concepts. What are they doing here, of all places?]
Combining frameworks and making analogies between your concepts is a good way of generating new ideas, but it’s far from rigorous and if anything encourages sloppiness if you’re not careful to recognize that it’s less formal reasoning and more another way of pointing to where the new things to learn might be.
6. Illusion of transparency
People will assume they know what you mean. You’ll assume you know what they mean in reply—that is, what a coincidence, that they know what you mean. They don’t.
7. Lifting the pot by one handle
The trouble with rationalist skills is that the opposite of every rationalist skill is also a rationalist skill.
In the comments to Epistemic Learned Helplessness (now there’s a name, eh?), komponisto writes:
Is it lotus eating or self-care? Should you reverse any advice you hear or not? Remembering a blogpost title isn’t going to tell you which situation you’re in; it’s at best a handle for noticing you’re in a situation where you might want to course correct following deeper analysis. But implicit in the assignment of handles is a claim that you ought to adjust more than you do in a certain direction. And you don’t want achieving a balance of opposite mistakes to depend on the relative catchiness of named concepts. When you lift the pot by one handle, the soup spills out.
(This isn’t, on its own, an argument against giving your ideas catchy names, so much as an argument for spending plenty of time discussing trade-offs and how to do the necessary deeper analysis. But recall that it’s still mostly the handle that gets remembered.)
8. Reification
You’ll tend to treat your named ideas as more concrete than they are, as meaning one particular thing to everyone, as having agency or causal powers, as unchanging and non-reframeable, as universal and context-independent, as territory, as binary, as objective.
(I wonder if this is related to a lot of our apparent confusion about double crux—people (including me) questioning its status as The Technique, while Duncan keeps trying to clarify how it’s not about executing an algorithm, it’s about internalizing what generates the algorithm; it’s not designed for use in earnest, it’s a pedagogical tool for practicing mental habits; it’s not about Double Crux, it’s about double-cruxiness.)
9. Rounding error
Conor again:
Or, well, the opposite of that, where you round everything off to ingroup-approved jargon.
***
You can’t stop titling your blog posts, but don’t force it, or you’ll end up with virality tracking the wrong things (more so than usual, anyway). In particular, maybe don’t optimize so much towards things like catchiness, metaphorical weight, uniqueness, and communicability—even for the sake of rapid, rigorous, distinction-rich, high-level discussion (or perhaps especially for that sake, since this kind of chunking is either unnecessary or destructive for all of these but speed)—over choosing words that prepare people to read and understand what you say.
(And even as far as speed of communication, what’s weird is that the rationalist writing stereotype is absurdly prolix—it feels like the compression isn’t happening where it matters, just where it feels powerful for hiding complex abstraction in a minimally intelligible way.)
You shouldn’t stop trying to draw more distinctions, but maybe you can do that without encouraging one particular way of pointing to your distinction to reify and metastasize.
You won’t stop sloppily compressing and combining complex ideas, but maybe worry more about unpacking things, giving examples, checking understanding, and asking questions.