If the goal is for conversations to be making epistemic progress, with the caveat that individual people have additional goals as well (such as obtaining or maintaining high status within their peer group), and Demon Threads “aren’t important” in the sense that they help neither of these goals, then it seems the solution would simply be better tricks participants in a discussion can use in order to notice when these are happening or likely to happen. But I think it’s pretty hard to actually measure how much status is up for grabs in a given conversation. I don’t think it’s literally zero—I remember who said what in a conversation and if they did or didn’t have important insights—but it’s definitely possible that different people come in with different weightings of importance of epistemic progress vs. being seen as intelligent or insightful. The key to the stickiness and energy-vacuum nature of the demon threads, I think, is that if social stakes are involved, they are probably zero-sum, or at least seen that way.
I have personally noticed that many of the candidate “Demon” threads contain a lot of specific phrases that sort of give away that social stakes are involved, and that there could be benefits to tabooing some of these phrases. To give some examples:
“You’re being uncharitable.”
“Arguing in bad faith.”
“Sincere / insincere.”
“This sounds hostile” (or other comments about tone or intent).
These phrases are usually but not always negative, as they can be used in a positive sense (i.e. charitable, good faith, etc.) but even in this case they are more often used to show support for a certain side, cheerleading, and so on. Generally, they have the characteristic of making a claim about or describing your opponent’s motives. How often is it actually necessary or useful to make such claims?
In the vast majority of situations, it is next to impossible to know the true motives of your debate partner or other conversation participants, and even in the best case scenario, poor models will be involved (combined with the fact that the internet tends to make this even more difficult). In addition, an important aspect of status games is that it is necessary to hide the fact that a status game is being played. Being “high-status” means that you are perceived as making an insightful and relevant point at the most opportune time. If someone in a conversation is being perceived as making status moves, that is equivalent to being perceived as low status. That means that the above phrases turn into weapons. They contain no epistemically useful information, and they are only being used to make the interaction zero-sum. Why would someone deliberately choose to make an interaction zero-sum? That’s a harder question, but my guess would be that it is a more aggressive tactic to get someone to back down from their position, or just our innate political instincts assuming the interaction is already zero-sum.
There is no need for any conversation to be zero-sum, necessarily. Even conversations where a participant is shown to be incorrect can lead to new insights, and so status benefits could even be conferred on the “losers” of these conversations. This isn’t denying social reality, it just means that it is generally a bad idea to make assumptions about someone else’s intent during a conversation, especially negative assumptions. I have seen these assumptions lead to a more productive discussion literally zero times.
So additional steps I might want to add:
Notice if you have any assumptions or models about your conversation partner’s intents. If yes—just throw them out. Even positive ones won’t really be useful, negative ones will be actively harmful.
Notice your own intents. It’s not wrong to want to gain some status from the interactions. But if you feel that if your partner wins, you lose, ask yourself why. Taking the conversation private might help, but you might also care about your status in the eyes of your partner, in which case turning the discussion private might not change this. Would a different framing or context allow you both to win?
Others have approached this from slightly different angles, but I’d say “you’re being uncharitable” is a symptom rather than a cause. If the conversation gets to the point where someone doesn’t trust their conversation partner, something has already gone wrong.
This strikes me as wrong/very optimistic. Distrust will be an inevitable concern for any online community that frequently recruits new members (because how are you supposed to already trust new members?)
“Something went wrong” is perhaps not the best way to phrase it, my point was more like: if you’re diagnosing something as wrong with the thread, you don’t solve the problem by preventing people from saying “you’re operating in bad faith”, you solve the problem by fixing the fact that people are operating in bad faith.
Hmm, I feel like we can make significant intellectual progress without everyone having to trust everyone else. And also don’t think there are that many interventions that reliably establish trust between parties that don’t just mostly consist of people being around each other for a while without getting into conflict.
Huh, really? I think there are fairly standard operating procedures for “how to converse in good faith”, that are pretty common in rationalist circles, and should be common/expected in rationalist circles, and if people are failing to live up to them I’d expect a given conversation to be less productive.
I think we might be using different definitions of “trust”, as a consequence of assigning different levels of importance to different aspects of the underlying concept.
I.e. I am thinking of trust as more something along the lines of “I expect the other person to actually have my well-being in mind”, whereas you might be pointing at one of the followng “I expect the other person is not going to accidentally hurt me/ doesn’t have an intention of hurting me/ is following a process that makes adversarial behavior inconvenient”
I think statements about models of a conversation partner’s intent can be good or bad. They are bad if they’re being used as accustations. They’re potentially good if they’re used in the context of a request for understanding (e.g. “I feel like your tone in this post is hostile—was that your intention?”) I don’t see the latter much outside of the LW-sphere, but when I do see it, I think it has value.
Hoo, boy, I think tabooing language that looks explicitly status-y is both a bad idea and won’t even get you what you want—anyone who really wants to do status stuff will just find more obfuscated language for doing it (including me).
I would probably like it if people went more in the NVC / Circling direction, away from claims about someone else and towards claims about themselves, e.g. “I feel frustrated” as opposed to “you’re being uncharitable,” but the way you get people to do this is not by tabooing or even by recommending tabooing.
Mostly I just want people to stop bringing models about the other person’s motives or intentions into conversations, and if tabooing words or phrases won’t accomplish that, and neither will explicitly enforcing a norm, then I’m fine not going that route. It will most likely involve simply arguing that people should adopt a practice similar to what you mentoned.
I propose that some people may say it because it is true and because they have a naive hope that the other party would try to be more charitable if they said it.
make the interaction zero-sum
All disagreements are zero sum, in the sense that one party is right and the other is wrong. A disagreement in only positive sum when your initial priors are so low that the other side only needs a few comments of text to provide sufficient information to change your mind, in other words, when you don’t know what you’re talking about. On the other hand, if you’ve already spent an hour in your life thinking about the topic, then you’ve probably already considered and dismissed the kinds of arguments the other side will bring up (and that’s assuming that you managed to explain what you view is well enough, so that their arguments are relevant to begin with).
Frankly, I’m bothered by how much you blame status games, while completely ignoring the serious challenges of identifying and resolving confusion.
I’m not really faulting all status games in general, only tactics which force them to become zero-sum. It’s basically unreasonable to ask that humans change their value systems so that status doesn’t play any role, but what we can do is alter the rules slightly so that outcomes we don’t like become improbable. If I’m accused of being uncharitable, I have no choice but to defend myself, because being seen as “an uncharitable person” is not something I want to be included in anyone’s models of me (even in the case where it’s true). Even in one-on-one coversations there’s no reason to disengage if this claim was made against me. Especially when it’s a person you trust or admire (more likely if it’s a private conversation) and therefore I care a lot what the other person thinks of me. That’s where the stickyness of demon threads comes from, where disengaging results in the loss of something for either party.
There’s a second type of demon thread where participants get dragged into dead ends that are very deep in, without a very clear map of where the conversation is heading. But I think these reduce to the ususal problems of identifying and resolving confusion, and can’t really be resolved by altering incentives / discussion norms.
It is bad to discuss abstract things. Do you agree that Kensho is an example of a demon thread? Is it a first type or second type? How about the subthread that starts here? I claim that it’s all “second type”. I claim that “first type”, status-game based demon threads without deep confusion, if they exist at all, aren’t even a problem to anyone. I claim that if, in a thread, there are both status games and deep confusion, the games are caused by the frustration resulting from the confusion, not the other way around. Confusion is the real root problem.
the ususal problems of identifying and resolving confusion
Are they “usual”, mundane problems? Do we know of any good solutions? Do we at least have past discussions about them?
<...> can’t really be resolved by altering incentives / discussion norms.
Confusion in the sense of one or both parties coming to the table with incorrect models is a root cause, but this is nearly always the default situation. We ostensibly partake in a conversation in order to update our models to more accurate ones and reduce confusion. So while yes, a lack of confusion would make bad conversations less likely, it also just reduces the need for the conversation to begin with.
And here we’re talking about a specific type of conversation that we’ve claimed is a bad thing and should be prevented. Here we need to identify a different root cause besides “confusion” which was too general of a root cause to explain these specific types of conversations.
What I’m claiming as a candidate cause is that there are usually other underlying motives for a conversation besides resolving disagreement. In addition people are bringing models of the other person’s confusion / motives in to the discussion, and that’s what I argue is causing problems and is a practice that should be set aside.
I think the Kensho post did spawn demon threads and that these threads contained the characteristics I mentioned in my original comment.
We can say that all disagreements start with confusion. Then I claim that if the confusion is quickly resolved, or if one of the parties exits the conversation, then the thread is normal and healthy. And that in all other cases the thread is demonic. Not all confusion is created equal. I’m claiming that the depth of this initial confusion is the best predictor of demon threads. I can understand why status games would prevent someone from exiting, but people ignoring their deep confusions is not a good outcome, so we don’t really want them to exit, we want them to resolve it. I don’t really see how status games could deepen the confusion.
In addition people are bringing models of the other person’s confusion / motives in to the discussion
I’d call that “confusion about what the other party thinks”, and put it under the umbrella of general confusion. In fact that’s the first kind of confusion I think about, when I think of demonic threads, but object-level confusion is important too. Maybe we aren’t disagreeing?
If the goal is for conversations to be making epistemic progress, with the caveat that individual people have additional goals as well (such as obtaining or maintaining high status within their peer group), and Demon Threads “aren’t important” in the sense that they help neither of these goals, then it seems the solution would simply be better tricks participants in a discussion can use in order to notice when these are happening or likely to happen. But I think it’s pretty hard to actually measure how much status is up for grabs in a given conversation. I don’t think it’s literally zero—I remember who said what in a conversation and if they did or didn’t have important insights—but it’s definitely possible that different people come in with different weightings of importance of epistemic progress vs. being seen as intelligent or insightful. The key to the stickiness and energy-vacuum nature of the demon threads, I think, is that if social stakes are involved, they are probably zero-sum, or at least seen that way.
I have personally noticed that many of the candidate “Demon” threads contain a lot of specific phrases that sort of give away that social stakes are involved, and that there could be benefits to tabooing some of these phrases. To give some examples:
“You’re being uncharitable.”
“Arguing in bad faith.”
“Sincere / insincere.”
“This sounds hostile” (or other comments about tone or intent).
These phrases are usually but not always negative, as they can be used in a positive sense (i.e. charitable, good faith, etc.) but even in this case they are more often used to show support for a certain side, cheerleading, and so on. Generally, they have the characteristic of making a claim about or describing your opponent’s motives. How often is it actually necessary or useful to make such claims?
In the vast majority of situations, it is next to impossible to know the true motives of your debate partner or other conversation participants, and even in the best case scenario, poor models will be involved (combined with the fact that the internet tends to make this even more difficult). In addition, an important aspect of status games is that it is necessary to hide the fact that a status game is being played. Being “high-status” means that you are perceived as making an insightful and relevant point at the most opportune time. If someone in a conversation is being perceived as making status moves, that is equivalent to being perceived as low status. That means that the above phrases turn into weapons. They contain no epistemically useful information, and they are only being used to make the interaction zero-sum. Why would someone deliberately choose to make an interaction zero-sum? That’s a harder question, but my guess would be that it is a more aggressive tactic to get someone to back down from their position, or just our innate political instincts assuming the interaction is already zero-sum.
There is no need for any conversation to be zero-sum, necessarily. Even conversations where a participant is shown to be incorrect can lead to new insights, and so status benefits could even be conferred on the “losers” of these conversations. This isn’t denying social reality, it just means that it is generally a bad idea to make assumptions about someone else’s intent during a conversation, especially negative assumptions. I have seen these assumptions lead to a more productive discussion literally zero times.
So additional steps I might want to add:
Notice if you have any assumptions or models about your conversation partner’s intents. If yes—just throw them out. Even positive ones won’t really be useful, negative ones will be actively harmful.
Notice your own intents. It’s not wrong to want to gain some status from the interactions. But if you feel that if your partner wins, you lose, ask yourself why. Taking the conversation private might help, but you might also care about your status in the eyes of your partner, in which case turning the discussion private might not change this. Would a different framing or context allow you both to win?
Others have approached this from slightly different angles, but I’d say “you’re being uncharitable” is a symptom rather than a cause. If the conversation gets to the point where someone doesn’t trust their conversation partner, something has already gone wrong.
This strikes me as wrong/very optimistic. Distrust will be an inevitable concern for any online community that frequently recruits new members (because how are you supposed to already trust new members?)
“Something went wrong” is perhaps not the best way to phrase it, my point was more like: if you’re diagnosing something as wrong with the thread, you don’t solve the problem by preventing people from saying “you’re operating in bad faith”, you solve the problem by fixing the fact that people are operating in bad faith.
Hmm, I feel like we can make significant intellectual progress without everyone having to trust everyone else. And also don’t think there are that many interventions that reliably establish trust between parties that don’t just mostly consist of people being around each other for a while without getting into conflict.
Huh, really? I think there are fairly standard operating procedures for “how to converse in good faith”, that are pretty common in rationalist circles, and should be common/expected in rationalist circles, and if people are failing to live up to them I’d expect a given conversation to be less productive.
I think we might be using different definitions of “trust”, as a consequence of assigning different levels of importance to different aspects of the underlying concept.
I.e. I am thinking of trust as more something along the lines of “I expect the other person to actually have my well-being in mind”, whereas you might be pointing at one of the followng “I expect the other person is not going to accidentally hurt me/ doesn’t have an intention of hurting me/ is following a process that makes adversarial behavior inconvenient”
Ah, yes. That is what I meant in this case.
I think statements about models of a conversation partner’s intent can be good or bad. They are bad if they’re being used as accustations. They’re potentially good if they’re used in the context of a request for understanding (e.g. “I feel like your tone in this post is hostile—was that your intention?”) I don’t see the latter much outside of the LW-sphere, but when I do see it, I think it has value.
Hoo, boy, I think tabooing language that looks explicitly status-y is both a bad idea and won’t even get you what you want—anyone who really wants to do status stuff will just find more obfuscated language for doing it (including me).
I would probably like it if people went more in the NVC / Circling direction, away from claims about someone else and towards claims about themselves, e.g. “I feel frustrated” as opposed to “you’re being uncharitable,” but the way you get people to do this is not by tabooing or even by recommending tabooing.
Mostly I just want people to stop bringing models about the other person’s motives or intentions into conversations, and if tabooing words or phrases won’t accomplish that, and neither will explicitly enforcing a norm, then I’m fine not going that route. It will most likely involve simply arguing that people should adopt a practice similar to what you mentoned.
I propose that some people may say it because it is true and because they have a naive hope that the other party would try to be more charitable if they said it.
All disagreements are zero sum, in the sense that one party is right and the other is wrong. A disagreement in only positive sum when your initial priors are so low that the other side only needs a few comments of text to provide sufficient information to change your mind, in other words, when you don’t know what you’re talking about. On the other hand, if you’ve already spent an hour in your life thinking about the topic, then you’ve probably already considered and dismissed the kinds of arguments the other side will bring up (and that’s assuming that you managed to explain what you view is well enough, so that their arguments are relevant to begin with).
Frankly, I’m bothered by how much you blame status games, while completely ignoring the serious challenges of identifying and resolving confusion.
I’m not really faulting all status games in general, only tactics which force them to become zero-sum. It’s basically unreasonable to ask that humans change their value systems so that status doesn’t play any role, but what we can do is alter the rules slightly so that outcomes we don’t like become improbable. If I’m accused of being uncharitable, I have no choice but to defend myself, because being seen as “an uncharitable person” is not something I want to be included in anyone’s models of me (even in the case where it’s true). Even in one-on-one coversations there’s no reason to disengage if this claim was made against me. Especially when it’s a person you trust or admire (more likely if it’s a private conversation) and therefore I care a lot what the other person thinks of me. That’s where the stickyness of demon threads comes from, where disengaging results in the loss of something for either party.
There’s a second type of demon thread where participants get dragged into dead ends that are very deep in, without a very clear map of where the conversation is heading. But I think these reduce to the ususal problems of identifying and resolving confusion, and can’t really be resolved by altering incentives / discussion norms.
It is bad to discuss abstract things. Do you agree that Kensho is an example of a demon thread? Is it a first type or second type? How about the subthread that starts here? I claim that it’s all “second type”. I claim that “first type”, status-game based demon threads without deep confusion, if they exist at all, aren’t even a problem to anyone. I claim that if, in a thread, there are both status games and deep confusion, the games are caused by the frustration resulting from the confusion, not the other way around. Confusion is the real root problem.
Are they “usual”, mundane problems? Do we know of any good solutions? Do we at least have past discussions about them?
Why not? This is not obvious to me.
Confusion in the sense of one or both parties coming to the table with incorrect models is a root cause, but this is nearly always the default situation. We ostensibly partake in a conversation in order to update our models to more accurate ones and reduce confusion. So while yes, a lack of confusion would make bad conversations less likely, it also just reduces the need for the conversation to begin with.
And here we’re talking about a specific type of conversation that we’ve claimed is a bad thing and should be prevented. Here we need to identify a different root cause besides “confusion” which was too general of a root cause to explain these specific types of conversations.
What I’m claiming as a candidate cause is that there are usually other underlying motives for a conversation besides resolving disagreement. In addition people are bringing models of the other person’s confusion / motives in to the discussion, and that’s what I argue is causing problems and is a practice that should be set aside.
I think the Kensho post did spawn demon threads and that these threads contained the characteristics I mentioned in my original comment.
We can say that all disagreements start with confusion. Then I claim that if the confusion is quickly resolved, or if one of the parties exits the conversation, then the thread is normal and healthy. And that in all other cases the thread is demonic. Not all confusion is created equal. I’m claiming that the depth of this initial confusion is the best predictor of demon threads. I can understand why status games would prevent someone from exiting, but people ignoring their deep confusions is not a good outcome, so we don’t really want them to exit, we want them to resolve it. I don’t really see how status games could deepen the confusion.
I’d call that “confusion about what the other party thinks”, and put it under the umbrella of general confusion. In fact that’s the first kind of confusion I think about, when I think of demonic threads, but object-level confusion is important too. Maybe we aren’t disagreeing?