Sometimes a false belief about a domain can be quite damaging, and a true belief can be quite valuable.
For example, suppose there is a 1000-person company. I tend to think that credit allocation for the success of the company is heavy tailed, and that there’s typically 1-3 people who the company just would zombify and die without, and ~20 people who have the key context and understanding that the 1-3 people can work with to do new and live things. (I’m surely oversimplifying because I’ve not ever been on the inside with a 1000-person company.) In this situation it’s very valuable to know who the people are who deserve the credit allocation. Getting the wrong 1-3 people is a bit of a disaster. This means that discussing it, raising hypotheses, bringing up bad arguments, bringing up arguments due to motivated cognition, and so on, can be unusually costly, and conversations about it can feel quite fraught.
Other fraught topics include breaking up romantically, quitting your job, leaving a community club or movement. I think taboo tradeoffs have a related feeling, like bringing up whether to lie in a situation, whether to cheat in a situation, or when to exchange money for values like honor and identity and dignity.
Other times getting an answer slightly wrong does not have this drop-off in value, and so discussion is less fraught. Non-fraught topics include where to go for dinner, cool things you can do with ChatGPT,
Broadly lots of things that are heavy-tailed in the realm of credit allocation feel this way to me. The sad thing is that cultures often feel to me totally against raising hypotheses in domains that are fraught, unless the person goes through a super costly counter-signalling procedure to signal that they’re doing it for the right reasons and not for the wrong ones.
Personally a bunch of conflict I’ve encountered has been from trying to use the sorts of reasoning tools and heuristics that I’d use for non-fraught subjects, for fraught-subjects. People rightly are pretty worried that something extremely bad will happen if I am even a little off or rely on a rough heuristic.
The domain that I think about this most at the minute is hiring/firing. I think this sort of breaking-of-relationships is pretty unnatural for humans i.e. this sort of thing didn’t happen regularly in the ancestral environment as part of the course of normal life, it was usually a life-ending disaster if you were kicked out of the tribe or if someone decided to backstab you. It’s something that all the human social alarm bells push back on. And yet it’s one of the most important decisions. My dream is to be able to discuss whether to fire people in an organization as straightforwardly as we would discuss whether to add a feature to a product, but this is nonstandard and far more fraught (and the experiences of firing people, even if they basically went well from most perspectives, have been some of the most emotionally wrecking experiences).
I don’t know what the right strategy is to have for being able to straightforwardly analyze and discuss ‘fraught’ topics. I mostly think it’s a cultural thing that you have to practice, and take opportunities where you can to have low-stakes discussions about fraught topics. This is one of the strongest reasons to get into heated arguments about minor things. It helps you negotiate over and agree on the principles involved, which you can then rely on when a major thing comes up.
I suspect the number/ratio of “key” personnel is highly variable, and in companies that aren’t sole-founder-plus-employees, there is a somewhat fractal tree of cultural reinforcement, where as long as there’s a sufficient preponderance of alignment at the level below the key person, the organization can survive the loss.
But that’s different from your topic—you want to know how to turn fraught high-stakes topics into simpler more legible discussions. I’m not sure that’s possible—the reason they’re fraught is the SAME as the reason it’s important to get it right. They’re high-stakes because they matter. And they matter because it affects a lot of different dimensions of the operation and one’s life, and those dimensions are entangled with each other BECAUSE of how valuable the relationship is to each side.
The sad thing is that cultures often feel to me totally against raising hypotheses in domains that are fraught, unless the person goes through a super costly counter-signalling procedure to signal that they’re doing it for the right reasons and not for the wrong ones.
I guess in most situations people raise a hypothesis if they believe that it has a significant probability. Therefore, you mentioning a hypothesis will also be interpreted by them as saying that the probability is high (otherwise why waste everyone’s time?).
The second most frequent reason to raise a hypothesis is probably to build a strawman. You must signal it clearly, to avoid possible misunderstanding.
Sometimes a false belief about a domain can be quite damaging, and a true belief can be quite valuable.
For example, suppose there is a 1000-person company. I tend to think that credit allocation for the success of the company is heavy tailed, and that there’s typically 1-3 people who the company just would zombify and die without, and ~20 people who have the key context and understanding that the 1-3 people can work with to do new and live things. (I’m surely oversimplifying because I’ve not ever been on the inside with a 1000-person company.) In this situation it’s very valuable to know who the people are who deserve the credit allocation. Getting the wrong 1-3 people is a bit of a disaster. This means that discussing it, raising hypotheses, bringing up bad arguments, bringing up arguments due to motivated cognition, and so on, can be unusually costly, and conversations about it can feel quite fraught.
Other fraught topics include breaking up romantically, quitting your job, leaving a community club or movement. I think taboo tradeoffs have a related feeling, like bringing up whether to lie in a situation, whether to cheat in a situation, or when to exchange money for values like honor and identity and dignity.
Other times getting an answer slightly wrong does not have this drop-off in value, and so discussion is less fraught. Non-fraught topics include where to go for dinner, cool things you can do with ChatGPT,
Broadly lots of things that are heavy-tailed in the realm of credit allocation feel this way to me. The sad thing is that cultures often feel to me totally against raising hypotheses in domains that are fraught, unless the person goes through a super costly counter-signalling procedure to signal that they’re doing it for the right reasons and not for the wrong ones.
Personally a bunch of conflict I’ve encountered has been from trying to use the sorts of reasoning tools and heuristics that I’d use for non-fraught subjects, for fraught-subjects. People rightly are pretty worried that something extremely bad will happen if I am even a little off or rely on a rough heuristic.
The domain that I think about this most at the minute is hiring/firing. I think this sort of breaking-of-relationships is pretty unnatural for humans i.e. this sort of thing didn’t happen regularly in the ancestral environment as part of the course of normal life, it was usually a life-ending disaster if you were kicked out of the tribe or if someone decided to backstab you. It’s something that all the human social alarm bells push back on. And yet it’s one of the most important decisions. My dream is to be able to discuss whether to fire people in an organization as straightforwardly as we would discuss whether to add a feature to a product, but this is nonstandard and far more fraught (and the experiences of firing people, even if they basically went well from most perspectives, have been some of the most emotionally wrecking experiences).
I don’t know what the right strategy is to have for being able to straightforwardly analyze and discuss ‘fraught’ topics. I mostly think it’s a cultural thing that you have to practice, and take opportunities where you can to have low-stakes discussions about fraught topics. This is one of the strongest reasons to get into heated arguments about minor things. It helps you negotiate over and agree on the principles involved, which you can then rely on when a major thing comes up.
I suspect the number/ratio of “key” personnel is highly variable, and in companies that aren’t sole-founder-plus-employees, there is a somewhat fractal tree of cultural reinforcement, where as long as there’s a sufficient preponderance of alignment at the level below the key person, the organization can survive the loss.
But that’s different from your topic—you want to know how to turn fraught high-stakes topics into simpler more legible discussions. I’m not sure that’s possible—the reason they’re fraught is the SAME as the reason it’s important to get it right. They’re high-stakes because they matter. And they matter because it affects a lot of different dimensions of the operation and one’s life, and those dimensions are entangled with each other BECAUSE of how valuable the relationship is to each side.
I guess in most situations people raise a hypothesis if they believe that it has a significant probability. Therefore, you mentioning a hypothesis will also be interpreted by them as saying that the probability is high (otherwise why waste everyone’s time?).
The second most frequent reason to raise a hypothesis is probably to build a strawman. You must signal it clearly, to avoid possible misunderstanding.
This is great. Encouragement to turn it into a top level post if you want it.