Most people don’t learn jargon by reading the original source for a term or phrase, they learn it from other people. Therefore one of the best ways to stop your jargon from being misused is to coin it in such a way that the jargon is a compressed representation of the concept it refers to. Authors in this milieu tend to be really bad at this. You yourself wrote about the concept of a ‘demon thread’, which I would like to (playfully) nominate for worst jargon ever coined on LessWrong. Its communicated meaning without the original thread boils down to ‘bad thread’ or ‘unholy thread’, which means that preserving the meaning you wanted it to have is a multi-front uphill battle in snow.
Another awful example from the CFAR handbook is the concept of ‘turbocharging’, which is a very specific thing but the concept handle just means ‘fast electricity’ or ‘speedy movement’. Were it not for the context, I wouldn’t know it was about learning at all. Even when I do have that context, it isn’t clear what makes it ‘turbo’. If it were more commonly used it would be almost instantly diluted without constant reference back to the original source.
For a non-LessWrong example, consider the academic social justice concept of ‘privilege’, which has (or had) a particular meaning that was useful to have a word for. However mainstream political commentary has diluted this phrase almost to the point of uselessness, making it a synonym for ‘inequality’.
It’d be interesting to do a study of say, 20-50 jargon terms and see how much level of dilution corresponds to degree-of-self-containment. In any case I suspect that trying to make jargon more self contained in its meaning would reduce misuse. “Costly Signaling” is harder to misinterpret than “Signaling”, for example.
Another option might be to use a word without any baggage. For example, Moloch seems to have held onto its original meaning pretty well but then maybe that’s because the source document is so well known.
That post is a fairly interesting counterargument, thanks for linking it. This passage would be fun to try out:
This prompted me to think that it might be valuable to buy a bunch of toys from a thrift store, and to keep them at hand when hanging out with a particular person or small group. When you have a concept to explore, you’d grab an unused toy that seemed to suit it decently well, and then you’d gesture with it while explaining the concept. Then later you could refer to “the sparkly pink ball thing” or simply “this thing” while gesturing at the ball. Possibly, the other person wouldn’t remember, or not immediately. But if they did, you could be much more confident that you were on the same page. It’s a kind of shared mnemonic handle.
My problem with s1 and s2 is that it’s very difficult to remember which is which unless you’ve had it reinforced a bunch of times to remember. I tend to prefer good descriptive names to nondescript ones, but certainly nondescriptive names are better than bad names which cause people to infer meaning that isn’t there.
On the s1/s2 thing, there are alternatives and I try to promote them when possible, especially since around these parts people tend to use s1/s2 for a slightly different but related purpose to their original formulation anyway. The alternative names for the clusters (not all the source names line up exactly, though):
The fact that there are subtly different purposes for the alternative naming schema could be a strength.
If I’m talking about biases I might talk about s1/s2. If I’m talking about motivation I might go for elephant/rider. If I’m talking about adaptations being executed I’d probably use blue minimising robot/side module.
I’m not sure whether others do something similar but I find the richness of the language helpful to distinguish in my own mind the subtly different dichotomies which are being alluded to.
Data point: I remember that System 1 is the fast, unconscious process by associating it with firstness—it’s more primal than slow thinking. This is probably somewhat true, but it defeats the purpose (?).
Yes, I frequently violate this myself, but at least I’ve been trying to keep it down.
If you do violate the rule, then make the new word as self-explanatory as possible. “Seed AI”, “Friendly AI”, and “neurohacking” are good. “External reference semantics” is bad.
Unfortunately following Eliezer’s advice seems to perhaps do the most to create the issues being considered about jargon here, because the more readily comprehensible jargon seems on first hearing it the more likely it is that it will be misremember and misapplied later (though “Schelling point” seems a notable case of something with no false-friend interpretation that gets misused anyway).
I do quite agree on the “the best jargon is self explanatory” thing, just noting that it’s often fairly hard. (I’m interested if you have alternate suggestions for demon thread, although fwiw I find “unholy thread” a bit more intuitive than ‘uphill battle in snow’, since there’s a lot of reasons something might be like an uphill battle in snow, and one feature of the demon thread is ‘everyone is being subtly warped into more aggressive, hostile versions of themselves’. I agree that connotation still pretty culture dependent though)
“Uphill battle” is a standard English idiom, such idioms are often fairly nonsensical if you think about them hard enough (e.g, “have your cake and eat it too”), but they get a free pass because everyone knows what they mean.
and one feature of the demon thread is ‘everyone is being subtly warped into more aggressive, hostile versions of themselves’
See that’s obvious in your mind, but I don’t think it’s obvious to others from the phrase ‘demon thread’. In fact, hearing it put like that the name suddenly makes much more sense! However, it would never be apparent to me from hearing the phrase. I would go for something like “Escalation Spiral” or “Reciprocal Misperception” or perhaps “Retaliation Bias”.
One thing I like to do before I pick a phrase in this vein, is take the most likely candidates and do a survey with people I know where I ask them, before they know anything else, what they think when they hear the phrase. That’s often steered me away from things I thought conveyed the concept well but actually didn’t.
It’s importantly different from a flame war – flame war implies things are already gone to hell, and people are all-out hostile at each other.
Escalation Spiral feels closest to what I was aiming for there (although it still feels a bit off to me, or at least I feel like I have a harder time using it in sentences for some reason. It felt kind of important to have the word “thread” in there, or to refer more directly to a forum discussion in some way)
The key point of a demon thread/escalation spiral/whatever is that it means things are subtly but noticeably bending towards confusion and hostility, even when everyone is well intentioned and on the same side, and you can see it happening in advance but it’s still real hard to do anything about.
If your product has subtle differences from existing products, that’s not a benefit. To buyers it’s a cost, and your product is supposed to have some benefit that compensates for that cost. For new words, that benefit is usually clarity, but the words “demon thread” are the opposite of clarity.
The whole point of jargon is to point to fine distinctions in things IMO
(not defending “demon thread” as a term, just the necessity of having a phrase for that concept. If I imagine calling a given LW demon thread a “flame war” I imagine people being like ’huh? it’s not a flame war?”)
Not objecting to the concept—having more concepts is good. But I think if you want to contribute to language, concepts are less than half of the work. Most of the work is finding the right words and making them work well with other words. Here’s a programming analogy: if you come up with a cool new algorithm and want to add it to a system that already has a billion lines of code, most of your effort should be spent on integrating with the system. Otherwise the whole system becomes crap over time. That’s how I think about these things: coining an ugly new word is affixing an ugly shed to the cathedral of language.
“Escalation spiral” is mixing two spatial metaphors, both far removed from the thing we’re talking about. That’s too abstract for me: being in a bad online argument doesn’t feel like walking up a spiral staircase. I prefer words that say how I feel about the thing—something like “quarreling”, “petty disagreement”, or “argumentative black hole”.
And for several months before writing the demon thread post, the entire ontology of how I thought about online discussion depended heavily on the demon-thread concept (which I still think is quite important). So, whenever I’d explain why I thought a given interaction was going poorly or how to improve it, I’d first have to explain a bunch of relevant concepts about the ontology, which made it harder to have a conversation.
I don’t know how much double-illusion of transparency has been going on. Maybe a lot. But my impression is that I’m now able to refer to “Demon Threads” as a small bite sized chunk of an argument, and
a) many people in the discussion have read the post, and even if the phrase was unintuitive to them, they know enough of what it means for me to make my point
b) the worst case scenario is that they think it means “bad thread”, which is, in fact, often good enough. (and in situations where the precise mechanics of demon threads matter, if the subject comes up and people seem to be missing subtleties there’s a post I can link to now that explains it in more detail)
[edit: I actually think it’s an important enough subgoal of the term to gracefully degrade into “bad thread”, which I’m less confident escalation spiral does, although unsure]
(I’ve updated the original demon thread article to begin with a much more succinct and hopefully clear definition at the top. I’ll try out “escalation spiral” and similar terms in conversation and see if they feel like they work, and consider updating the name)
For what it’s worth, I don’t feel like ‘escalation spiral’ is particularly optimal. The concept you’re going for is hard to compress into a few words because there are so many similar things. It was just the best I could come up with without spending a few hours thinking about it.
Most people don’t learn jargon by reading the original source for a term or phrase, they learn it from other people. Therefore one of the best ways to stop your jargon from being misused is to coin it in such a way that the jargon is a compressed representation of the concept it refers to. Authors in this milieu tend to be really bad at this. You yourself wrote about the concept of a ‘demon thread’, which I would like to (playfully) nominate for worst jargon ever coined on LessWrong. Its communicated meaning without the original thread boils down to ‘bad thread’ or ‘unholy thread’, which means that preserving the meaning you wanted it to have is a multi-front uphill battle in snow.
Another awful example from the CFAR handbook is the concept of ‘turbocharging’, which is a very specific thing but the concept handle just means ‘fast electricity’ or ‘speedy movement’. Were it not for the context, I wouldn’t know it was about learning at all. Even when I do have that context, it isn’t clear what makes it ‘turbo’. If it were more commonly used it would be almost instantly diluted without constant reference back to the original source.
For a non-LessWrong example, consider the academic social justice concept of ‘privilege’, which has (or had) a particular meaning that was useful to have a word for. However mainstream political commentary has diluted this phrase almost to the point of uselessness, making it a synonym for ‘inequality’.
It’d be interesting to do a study of say, 20-50 jargon terms and see how much level of dilution corresponds to degree-of-self-containment. In any case I suspect that trying to make jargon more self contained in its meaning would reduce misuse. “Costly Signaling” is harder to misinterpret than “Signaling”, for example.
Another option might be to use a word without any baggage. For example, Moloch seems to have held onto its original meaning pretty well but then maybe that’s because the source document is so well known.
EDIT: I see The sparkly pink ball thing makes a similar point.
That post is a fairly interesting counterargument, thanks for linking it. This passage would be fun to try out:
My problem with s1 and s2 is that it’s very difficult to remember which is which unless you’ve had it reinforced a bunch of times to remember. I tend to prefer good descriptive names to nondescript ones, but certainly nondescriptive names are better than bad names which cause people to infer meaning that isn’t there.
On the s1/s2 thing, there are alternatives and I try to promote them when possible, especially since around these parts people tend to use s1/s2 for a slightly different but related purpose to their original formulation anyway. The alternative names for the clusters (not all the source names line up exactly, though):
s1: near, concrete, id, fast, yin, hot, elephant, unconscious, machine, outside
s2: far, abstract, superego, slow, yang, cold, rider, conscious, monkey/homunculus, inside
I think near/far the best, but I think we’re stuck with s1/s2 at this point due to momentum.
The fact that there are subtly different purposes for the alternative naming schema could be a strength.
If I’m talking about biases I might talk about s1/s2. If I’m talking about motivation I might go for elephant/rider. If I’m talking about adaptations being executed I’d probably use blue minimising robot/side module.
I’m not sure whether others do something similar but I find the richness of the language helpful to distinguish in my own mind the subtly different dichotomies which are being alluded to.
Data point: I remember that System 1 is the fast, unconscious process by associating it with firstness—it’s more primal than slow thinking. This is probably somewhat true, but it defeats the purpose (?).
Eliezer also mentioned this in his old article on writing advice:
Unfortunately following Eliezer’s advice seems to perhaps do the most to create the issues being considered about jargon here, because the more readily comprehensible jargon seems on first hearing it the more likely it is that it will be misremember and misapplied later (though “Schelling point” seems a notable case of something with no false-friend interpretation that gets misused anyway).
You mean to say that deliberate anti-epistemology, which combines dehumanization with anthropomorphism, turns out to be bad?
I do quite agree on the “the best jargon is self explanatory” thing, just noting that it’s often fairly hard. (I’m interested if you have alternate suggestions for demon thread, although fwiw I find “unholy thread” a bit more intuitive than ‘uphill battle in snow’, since there’s a lot of reasons something might be like an uphill battle in snow, and one feature of the demon thread is ‘everyone is being subtly warped into more aggressive, hostile versions of themselves’. I agree that connotation still pretty culture dependent though)
“Uphill battle” is a standard English idiom, such idioms are often fairly nonsensical if you think about them hard enough (e.g, “have your cake and eat it too”), but they get a free pass because everyone knows what they mean.
See that’s obvious in your mind, but I don’t think it’s obvious to others from the phrase ‘demon thread’. In fact, hearing it put like that the name suddenly makes much more sense! However, it would never be apparent to me from hearing the phrase. I would go for something like “Escalation Spiral” or “Reciprocal Misperception” or perhaps “Retaliation Bias”.
One thing I like to do before I pick a phrase in this vein, is take the most likely candidates and do a survey with people I know where I ask them, before they know anything else, what they think when they hear the phrase. That’s often steered me away from things I thought conveyed the concept well but actually didn’t.
Flame war. Don’t invent new words ;-)
It’s importantly different from a flame war – flame war implies things are already gone to hell, and people are all-out hostile at each other.
Escalation Spiral feels closest to what I was aiming for there (although it still feels a bit off to me, or at least I feel like I have a harder time using it in sentences for some reason. It felt kind of important to have the word “thread” in there, or to refer more directly to a forum discussion in some way)
The key point of a demon thread/escalation spiral/whatever is that it means things are subtly but noticeably bending towards confusion and hostility, even when everyone is well intentioned and on the same side, and you can see it happening in advance but it’s still real hard to do anything about.
If your product has subtle differences from existing products, that’s not a benefit. To buyers it’s a cost, and your product is supposed to have some benefit that compensates for that cost. For new words, that benefit is usually clarity, but the words “demon thread” are the opposite of clarity.
Is “escalation spiral” opposite of clarity?
The whole point of jargon is to point to fine distinctions in things IMO
(not defending “demon thread” as a term, just the necessity of having a phrase for that concept. If I imagine calling a given LW demon thread a “flame war” I imagine people being like ’huh? it’s not a flame war?”)
Not objecting to the concept—having more concepts is good. But I think if you want to contribute to language, concepts are less than half of the work. Most of the work is finding the right words and making them work well with other words. Here’s a programming analogy: if you come up with a cool new algorithm and want to add it to a system that already has a billion lines of code, most of your effort should be spent on integrating with the system. Otherwise the whole system becomes crap over time. That’s how I think about these things: coining an ugly new word is affixing an ugly shed to the cathedral of language.
Still curious if “escalation spiral” feels more or less clear.
Also wanted to flag that I think your most recent argument seems quite different from your initial one (i.e. “flame war. don’t invent new words.”)
“Escalation spiral” is mixing two spatial metaphors, both far removed from the thing we’re talking about. That’s too abstract for me: being in a bad online argument doesn’t feel like walking up a spiral staircase. I prefer words that say how I feel about the thing—something like “quarreling”, “petty disagreement”, or “argumentative black hole”.
And for several months before writing the demon thread post, the entire ontology of how I thought about online discussion depended heavily on the demon-thread concept (which I still think is quite important). So, whenever I’d explain why I thought a given interaction was going poorly or how to improve it, I’d first have to explain a bunch of relevant concepts about the ontology, which made it harder to have a conversation.
I don’t know how much double-illusion of transparency has been going on. Maybe a lot. But my impression is that I’m now able to refer to “Demon Threads” as a small bite sized chunk of an argument, and
a) many people in the discussion have read the post, and even if the phrase was unintuitive to them, they know enough of what it means for me to make my point
b) the worst case scenario is that they think it means “bad thread”, which is, in fact, often good enough. (and in situations where the precise mechanics of demon threads matter, if the subject comes up and people seem to be missing subtleties there’s a post I can link to now that explains it in more detail)
[edit: I actually think it’s an important enough subgoal of the term to gracefully degrade into “bad thread”, which I’m less confident escalation spiral does, although unsure]
(I’ve updated the original demon thread article to begin with a much more succinct and hopefully clear definition at the top. I’ll try out “escalation spiral” and similar terms in conversation and see if they feel like they work, and consider updating the name)
For what it’s worth, I don’t feel like ‘escalation spiral’ is particularly optimal. The concept you’re going for is hard to compress into a few words because there are so many similar things. It was just the best I could come up with without spending a few hours thinking about it.
Seconded; this interpretation didn’t ever occur to me before reading Raemon’s comment just now.