Can you clarify which bit was off-putting? The fact that any norms were being promoted or the specific norms being promoted?
Only the latter. And also the vehemence with which these viewpoints seemed to be held and defended. I got the impression that statements of the sort “yay truth as the only sacred value” received strong support; personally I find that off-putting in many contexts.
Edit: The reason I find it off-putting isn’t that I disagree with the position as site policy. More that sometimes the appropriate thing in a situation isn’t just to respond with some tirade about why it’s good to have an unempathetic site policy.
To give some more context: Only the first instance of this had to do with explicit calls for forum policy. This was probably the same example that inspired the dialogue between Jill and John above.
The second example was a comment on the question of making downvotes less salient. While I agree that the idea has drawbacks, I was a bit perplexed that a comment arguing against it got strongly upvoted despite including claims that felt to me like problematic “rationality for rationality’s sake”: Instead of allowing people to only look at demotivating information at specific times, we declare it antithetical to the “core of rationality” to hide information whether or not it overall makes people accomplish their goals better.
The third instance was an exchange you had about conversational tone and (lack of) charity. Toward the end you said that you didn’t like the way you phrased your initial criticism, but my quick impression (and I probably only skimmed the lengthy exchange and also don’t remember details) was that I generally thought your points seemed pretty defensible, and the way your conversation partner commented would have also thrown me off. “Tone and degree of charity are very important too” is a perspective I’d like to see represented more among LW users. (But if I’m in the minority, that’s fine and I don’t object to communities keeping their defining features if the majority feels that they are benefitting.)
That doesn’t feel true to me.
Maybe I expressed it poorly, but what I meant was just that rationality is not an end in itself. If I complain that some piece of advice is not working for me because it makes me (all-things-considered, long-term) less productive (towards the things that are most important to me) and less happy, and my conversation partner makes some unqualified statement to the degree of “but it’s rational to follow this type of advice”, I will start to suspect that they are misunderstanding what rationality isfor.
And also the vehemence with which these viewpoints seemed to be held and defended.
I agree there’s something like vehemence and it’s made all the conversations unpleasant and stressful. Someone countered to me that if you perceive someone to be threatening the very integrity of your ability to have conversations, it’s appropriate to break frame and get up in arms. I’m not convinced it’s warranted here, but maybe...
“Tone and degree of charity are very important too” is a perspective I’d like to see represented more among LW users. (But if I’m in the minority, that’s fine and I don’t object to communities keeping their defining features if the majority feels that they are benefitting.)
I’m not sure about the exact proportion of people’s perspectives. There definitely is a cluster of people (myself included) who think “tone”, etc. are significant. (This group also might be more averse to getting into online conflicts.) I’m also concerned about the number of people who would counterfactually engage more on LessWrong, except they dislike the conversations they’ll end up in currently.
There are a bunch of conversations going on about the topic right (some in semi-private which might be public soonish). There’s support (at least on the LW team) for an Archipelago type solution where people can opt-in into one of 2 or 3 norm sets. (Though that doesn’t quite fix site-level things like the karma notifier settings.) One of those spaces should have much more “civility.”
Maybe I expressed it poorly, but what I meant was just that rationality is not an end in itself.
Yeah, that’s reasonable. I think that many people, while agreeing with that (or something close to it), get very afraid as soon as someone says it that because they fear it’s going to be used to justify distinctly not-rational/damages the whole endeavor of being rational. I have some of this fear myself.
It seems to me that rationality is extremely fragile and vulnerable, such that even though rationality might serves other goals, you have to be very uncompromising with regards to rationality, especially core things like hiding information from yourself (I was lightly opposed to the negative karma hiding myself) even if it that has appararant costs.
But it’s hard. I think there are tricky questions to answer, but the conversation currently can be civil/happen without vehemence.
There are a bunch of conversations going on about the topic right (some in semi-private which might be public soonish).
Cool! And I appreciate the difficulty of the task at hand. :)
When I model these conversations, one failure mode I’m worried about is that the “more civility” position gets lumped together with other things that Lesswrong is probably right to be scared of.
So, the following is to delineate my own views from things I’m not saying:
I could imagine being fine with Bridgewater culture in many (but not all) contexts. I hate that in “today’s climate” it is difficult to talk about certain topics. I think it’s often the case that people complaining about tone or about not feeling welcome shouldn’t expect to have their needs accommodated.
And yet I still find some features of what I perceive to be “rationalist culture” very off-putting.
I don’t think I phrased it as well in my first comment, but I can fully get behind what Raemon said elsewhere in this thread:
Some of the language about “holding truth sacred” [...] has came across to me with a tone of single-minded focus that feels like not being willing to put an upper bound on a heart transplant, rather than earnestly asking the question “how do we get the most valuable truthseeking the most effective way?”
So it’s not that I’m saying that I’d prefer a culture where truth-seeking is occasionally completely abandoned because of some other consideration. Just that the side that superficially looks more virtuous when it comes to truth-seeking (for instance because they boldly proclaim the importance of not being bothered by tone/tact, downvote notifications, etc.) isn’t automatically what’s best in the long run.
Edited to add: I admit it’s a delicate balance to walk. But sometimes, people are inconsiderate in a way that definitely harms discussions. The principle of charity isn’t just a thing in philosophy to make people feel good; there’s also some methodological use to it. Likewise with trying to understand that other people have different minds from one’s own. There has to be a way to point out inconsiderateness that doesn’t get met with a response a la “tact doesn’t matter because truth is the only virtue.”
It seems to me that rationality is extremely fragile and vulnerable, such that even though rationality might serves other goals, you have to be very uncompromising with regards to rationality, especially core things like hiding information from yourself (I was lightly opposed to the negative karma hiding myself) even if it that has appararant costs.
I agree with that. But people can have very different psychologies. Most people are prone to overconfidence, but some people are underconfident and beat themselves up too much over negative feedback. If the site offers an optional feature that is very useful for people of the latter type, it’s at least worth considering whether that’s an overall improvement. I wasn’t even annoyed that people didn’t like the feature; it was more about the way in which the person argued. Generally, more display of awareness of people having different psychologies would please me. :)
I got the impression that statements of the sort “yay truth as the only sacred value” received strong support; personally I find that off-putting in many contexts.
I also find it off-putting in many contexts—perhaps most contexts. But if there’s any consequentialist value in having one space in the entire world where (within the confines of that space) truth is the only sacred value, perhaps lesswrong.com is a Schelling point?
Something that I’m maybe able to put into words now:
The classical example of “sacred values run amok” in my mind is when you ask people how much money a hospital should spend on a heart transplant for a dying child. People try to dodge the question, avoiding trading off a sacred value for a mundane value. Despite the fact that money can buy hospital equipment that saves other lives.
It’s plausible that hospital should hold “keeping people healthy and alive” as an overall sacred value, which they never trade off against. This might forbid some paths where resources are spent on things that weren’t necessary to keep people healthy and alive. But it doesn’t tell you what are the best strategies to go about it are. You’re allowed to sacrifice a boy’s life to buy hospital equipment. You’re even allowed to sacrifice a boy’s life to make sure your employees are well rested and not overly stressed. Running a hospital is a marathon, not a sprint.
Over the past couple years, I have updated to “yes, LessWrong should be the place focused on truthseeking.” I think I came to believe that right around the time I wrote Tensions in Truthseeking, in the process of writing the paragraph about instrumental sacredness. But that tells us what the question is, not what the answer is.
Some of the language about “holding truth sacred” (things you’ve said, and others) has came across to me with a tone of single-minded focus that feels like not being willing to put an upper bound on a heart transplant, rather than earnestly asking the question “how do we get the most valuable truthseeking the most effective way?”
There’s also the bit where operationalization matters. “Minimize falsehood” is a different function than “maximize true, good ideas over time” which is a different function than “maximize true, good ideas that are communicated well enough to impact the world.”
I definitely agree that there could exist perverse situations where there are instrumental tradeoffs to be made in truthseeking of the kind I and others have been suspicious of. For lack of a better term, let me call these “instrumentally epistemic” arguments: claims of the form, “X is true, but the consequences of saying it will actually result in less knowledge on net.” I can totally believe that some instrumentally epistemic arguments might hold. There’s nothing in my understanding of how the universe works that would prevent that kind of scenario from happening.
But in practice, with humans, I expect that a solid supermajority of real-world attempts to explicitly advocate for norm changes on “instrumentally epistemic” grounds are going to be utterly facilerationalizations with the (typically unconscious) motivation of justifying cowardice, intellectual dishonesty, ego-protection, &c.
I (somewhat apologetically) made an “instrumentally epistemic” argument in a private email thread recently, and Ben seemed super pissed in his reply (bold italics, incredulous tone, ”?!?!?!?!?!” punctuation). But the thing is—even if I might conceivably go on to defend a modified form of my original argument—I can’t blame Ben for using a pissed-off tone in his reply. “Instrumentally epistemic” arguments are an enormous red flag—an infrared flag thirty meters wide. Your prior should be that someone making an “instrumentally epistemic” argument can be usefully modeled as trying to undermine your perception of reality and metaphorically slash your tires (even if their conscious phonological loop never contains the explicit sentence “And now I’m going to try to undermine Ray Arnold’s perception of reality”).
Now, maybe that prior can be overcome for some arguments and some arguers! But the apparent failure of the one making the “instrumentally epistemic” argument to notice the thirty-meter red flag, is another red flag.
I don’t think the hospital example does the situation justice. The trade-off of choosing whether to spend money on a heart transplant or nurse salaries doesn’t seem analogous to choosing between truth and the occasional allegedly-instrumentally-epistemic lie (like reassuring your interlocutor that you respect them even when you don’t, in fact, respect them). Rather, it seems more closely analogous to choice of inquiry area (like whether to study truths about chemistry, or truths about biology), with “minutes of study time” as the resource to be allocated rather than dollars.
If we want a maximally charitable medical analogy for “instrumentally epistemic” lies, I would instead nominate chemotherapy, where we deliberately poison patients in the hope of hurting cancer cells more than healthy cells. Chemotherapy can be good if there’s solid evidence that you have a specific type of cancer that responds well to that specific type of chemotherapy. But you should probably check that people aren’t just trying to poison you!
I’m not advocating lying here, I’m advocating learning the communication skills necessary to a) actually get people to understand your point (which they’ll have a harder time with if they’re defensive), and b) not wasting dozens of hours unnecessarily (which could be better spent on figuring other things out).
[and to be clear, I also advocate gaining the courage to speak the truth even if your voice trembles, and be willing to fight for it when it’s important. Just, those aren’t the only skills a rationalist or a rationalist space needs. Listening, communicating clearly, avoiding triggering people’s “use language as politics mode”, and modeling minds and frames different from your own are key skills too]
Your prior should be that someone making an “instrumentally epistemic” argument can be usefully modeled as trying to undermine your perception of reality and metaphorically slash your tires (even if their conscious phonological loop never contains the explicit sentence “And now I’m going to try to undermine Ray Arnold’s perception of reality”).
Why do you think this prior is right?
But the apparent failure of the one making the “instrumentally epistemic” argument to notice the thirty-meter red flag, is another red flag.
This seems true only if your prior is so obviously right that one couldn’t disagree with it in good faith. I’m not convinced of this.
(As I mentioned I’m sympathetic to both sides of the debate here, but I find myself wanting to question your side more, because it seems to display a lot more certainty (along with associated signals such as exasperation and incredulity), which doesn’t seem justified to me.)
Thanks, I appreciate it a lot! You should be questioning my “side” as harshly as you see fit, because if you ask questions I can’t satisfactorily answer, then maybe my side is wrong, and I should be informed of this in order to become less wrong.
Why do you think this prior is right?
The mechanism by which saying true things leads to more knowledge is at least straightforward: you present arguments and evidence, and other people evaluate those arguments and evidence using the same general rules of reasoning that they use for everything else, and hopefully they learn stuff.
In order for saying true things to lead to less knowledge, we need to postulate some more complicated failure mode where some side-effect of speech disrupts the ordinary process of learning. I can totally believe that such failure modes exist, and even that they’re common. But lately I seem to be seeing a lot of arguments of the form, “Ah, but we need to coordinate in order to create norms that make everyone feel Safe, and only then can we seek truth.” And I just … really have trouble taking this seriously as a good faith argument rather than an attempt to collude to protect everyone’s feelings? Like, telling the truth is not a coordination problem? You can just unilaterally tell the truth.
associated signals such as exasperation and incredulity
But lately I seem to be seeing a lot of arguments of the form, “Ah, but we need to coordinate in order to create norms that make everyone feel Safe, and only then can we seek truth.” And I just … really have trouble taking this seriously as a good faith argument rather than an attempt to collude to protect everyone’s feelings?
I want to address something that I think is quite important in the context of this post, because I think you’re pattern matching the “let’s make a space where people’s needs are addressed,” to the standard social justice safe space, but there are actually 3 types of safe spaces, and the one you’re imagining is not related to the ones this post is talking about.
The social justice kind, where nobody is allowed to bring up arguments that make you feel unsafe, is the one you’re talking about. “We need to make everyone feel safe and can’t seek truth until we do that” is describing an environment where truth seeking is basically impossible. I think private spaces like that are important in a rationalist environment, because some people are fragile and need to heal before they can participate in truth seeking, but are almost never right for an organization that has the goal of seeking truth.
Then there’s the kind that this post is talking about. In this type of environment, it’s safe to say “This conversation is making me feel unsafe, so I need to leave”. It’s also safe to say “It feels like your need for safety is getting in the way of truthseeking” as well as for other people to push back on that if they think that this person’s need for safety is so great in this moment that we need to accommodate them for a bit and return to this topic later. I think the majority of public truth-seeking spaces would be served by adopting this type of safety, in lieu of something like Crocker’s rules.
Then there’s the third type of safe space. In this type of safe space, you can say “This topic is making me feel unsafe” and the expected response is “Awesome, then we’re going to keep throwing you in as many situations like this as possible, poke that emotional wound, and help you work through it so you can level up as an individual and we can level up as an organization.” In this case, the safety comes from the strict vetting procedures and strong culture that let you know that people poking your are sincere and skilled, and the people being poked have the emotional strength to deal with it. I think that a good majority of PRIVATE truth seeking spaces should strive to be this third type of safe space.
One of the mistakes I made in this post was conflate the second and third types of safe spaces, so for instance I posited a public space that also had radical transparency, which is really only a tool you should use in a culture with strong vetting. However, I definitely was not suggesting the first type of safe space, but I get the impression that that’s what you keep imagining.
In this type of environment, it’s safe to say “This conversation is making me feel unsafe, so I need to leave”.
I mean, in the case of a website that people use in their free time, you don’t necessarily even need an excuse: if you don’t find a conversation valuable (because it’s making you feel unsafe or for any other reason), you can just strong-downvote them and stop replying.
There was a recent case on Less Wrong where one of two reasons I gave for calling for end-of-conversation was that I was feeling “emotionally exhausted”, which seems similar to feeling unsafe. But that was me explaining why I didn’t feel like talking anymore. I definitely wasn’t saying that my interlocutor should give equal weight to his needs, my needs, and the needs of the forum of the whole. I don’t see how anyone is supposed to compute that.
I don’t see how anyone is supposed to compute that.
If your primary metaphor for thought is simple computations or mathematical functions, I can see how this would be very confusing, but I don’t think that’s actually the native architecture of our brains. Instead our brain is noticing patterns, creating reusable heuristics, and simulating other people using empathy.
When you look at the question using that native architecture, it becomes relatively simple to find a reasonable answer. This is the same way that we regularly find solutions to complex negotiations between multiple parties, or plan complex situations with multiple constraints, even though many of those tasks are naively uncomputable. The shared values and culture serve to make sure those heuristics are calibrated similarly between people.
Reply
When you look at the question using that native architecture, it becomes relatively simple to find a reasonable answer.
I don’t think “reasonable” is the correct word here. You keep assuming away the possibility of conflict. It’s easy to find a peaceful answer by simulating other people using empathy, if there’s nothing anyone cares about more than not rocking the boat. But what about the least convenient possible world where one party has Something to Protect which the other party doesn’t think is “reasonable”?
The shared values and culture serve to make sure those heuristics are calibrated similarly between people.
Riiiight, about that. The OP is about robust organizations in general without mentioning any specific organization, but given the three mentions of “truthseeking”, I’d like to talk about the special case of this website, and set it in the context of a previous discussion we’ve had.
I don’t think the OP is compatible with the shared values and culture established in Sequences-era Overcoming Bias and Less Wrong. I was there (first comment December 22, 2007). If the Less Wrong and “rationalist” brand names are now largely being held by a different culture with different values, I and the forces I represent have an interest in fighting to take them back.
Let me reply to your dialogue with another. To set the scene, I’ve been drafting a forthcoming post (working title: “Schelling Categories, and Simple Membership Tests”) in my nascent Sequence on the cognitive function of categories, which is to refer back to my post “The Univariate Fallacy”. Let’s suppose that by the time I finally get around to publishing “Schelling Categories” (like the Great Teacher, I suffer from writer’s molasses), the Jill from your dialogue has broken out of her simulation, instantiated herself in our universe, and joined the LW2 moderation team.
Jill: Zack, I’ve had another complaint—separate from the one in May—about your tendency to steer conversations towards divisive topics, and I’m going to ask you to tone it down a bit when on Frontpage posts.
Zack: What? Why? Wait, sorry—that was a rhetorical question, which I’ve been told is a violation of cooperative discourse norms. I think I can guess what motivated the complaint. But I want to hear you explain it.
Jill: Well, you mentioned this “univariate fallacy” again, and in the context of some things you’veTweeted, there was some concern that you were actually trying to allude to gender differences, which might make some community members of marginalized genders feel uncomfortable.
Zack: (aside) I’m guess I’m glad I didn’t keep calling it Lewontin’s fallacy.
(to Jill) So … you’re asking me to tone down the statistics blogging—on less wrong dot com—because some people who read what I write elsewhere can correctly infer that my motivation for thinking about this particular statistical phenomenon was because I needed it to help me make sense of an area of science I’ve been horrifiedlyfascinatedwith for the last fourteen years, and that scientific question might make some people feel uncomfortable?
Jill: Right. Truthseeking is very important. However, it’s clear that just choosing one value as sacred and not allowing for tradeoffs can lead to very dysfunctional belief systems. I believe you’ve pointed at a clear tension in our values as they’re currently stated: the tension between freedom of speech and truth, and the value of making a space that people actually want to have intellectual discussions at. I’m only asking you to give equal weight to your own needs, the needs of the people you’re interacting with, and the needs of the organization as a whole.
Zack: I said No. As a commenter on lesswrong.com, my duty and my only duty is to try to make—wait, scratch the “try”—to make contributions that advance the art of human rationality. I consider myself to have a moral responsibility to ignore the emotional needs of other commenters—and symmetrically, I think they have a moral responsibility to ignore mine.
Jill: I’d prefer that you be more charitable and work to steelman what I said.
Zack: If you think I’ve misunderstood what you’ve said, I’m happy to listen to you clarify whatever part you think I’m getting wrong. The point of the principle of charity is that people are motivated to strawman their interlocutors; reminding yourself to be “charitable” to others helps to correct for this bias. But to tell others to be charitable to you without giving them feedback about how, specifically, you think they’re misinterpreting what you said—that doesn’t make any sense; it’s like you’re just trying to mash an “Agree with me” button. I can’t say anything about what your conscious intent might be, but I don’t know how to model this behavior as being in good faith—and I feel the same way about this new complaint against me.
Zack: If by “contextualizing norms” you simply mean that what a speaker means needs to be partially understood from context, and is more than just what the sentence the speaker said means, then I agree—that’s just former Denver Broncos quarterback Brian Griese philosopher of language H. P. Grice’s theory of conversational implicature. But when I apply contextualizing norms to itself and look at the context around which “contextualizing norms” was coined, it sure looks like the entire point of the concept is to shut down ideologically inconvenient areas of inquiry. It’s certainly understandable. As far as the unwashed masses are concerned, it’s probably for the best. But it’s not what this website is about—and it’s not what I’m about. Not anymore. I am an aspiring epistemic rationalist. I don’t negotiate with emotional blackmailers, I don’t double-crux with Suicide Rock, and I’ve got Something to Protect.
Jill: (baffled) What could possibly incentivize you to be so unpragmatic?
I don’t think “reasonable” is the correct word here. You keep assuming away the possibility of conflict. It’s easy to find a peaceful answer by simulating other people using empathy, if there’s nothing anyone cares about more than not rocking the boat. But what about the least convenient possible world where one party has Something to Protect which the other party doesn’t think is “reasonable”?
Yes, if someone has values that are in fact incompatible with the culture of the organization, they shouldn’t be joining that organization. I thought that was clear in my previous statements, but it may in fact have not been. If every damn time their own values are at odds with what are best for the organization given its’ values, that’s an incompatible difference. They should either find a different organization, or try the archipeligo model. There are such thing as irreconcilable value differences.
I don’t think the OP is compatible with the shared values and culture established in Sequences-era Overcoming Bias and Less Wrong.
I agree. I think when that culture was established, the community was missing important concepts about motivated reasoning and truth seeking and chose values that were in fact not optimized for the ultimate goal of creating a community that could solve important problems.
I think it is in fact good to experiment with the norms you’re talking about from the original site, but I think many of those norms originally caused the site to decline and people to go elsewhere. Given my current mental models, I predict a site that uses those norms to make less intellectual progress than a similar site using my norms although I expect you to have the opposite intuition. As I stated in the introduction, the goal of this post was simply to make sure that those mental models were in discourse.
Re your dialogue: The main thing that I got from it was that you think a lot of the arguments in the OP are motivated reasoning and will lead to bad incentives. I also got that this is a subject you care a lot about.
I think when that culture was established, the community was missing important concepts about motivated reasoning and truth seeking
Can you be more specific? Can you name three specific concepts about motivated reasoning and truthseeking that you know, but Sequences-era Overcoming Bias/Less Wrong didn’t?
I think many of those norms originally caused the site to decline and people to go elsewhere.
I mean, that’s one hypothesis. In contrast, my model has been that communities congregate around predictable sources of high-quality writing, and people who can produce high-quality content in high volume are very rare. Thus, once Eliezer Yudkowsky stopped being active, and Yvain a.k.a. the immortal Scott Alexander moved to Slate Star Codex (in part so that he could write about politics, which we’ve traditionally avoided), all the “intellectual energy” followed Scott to SSC.
Can you think of any testable predictions (or retrodictions) that would distinguish my model from your model?
I also got that this is a subject you care a lot about.
Can you be more specific? Can you name three specific concepts about motivated reasoning and truthseeking that you know, but Sequences-era Overcoming Bias/Less Wrong didn’t?
Here are a few:
The importance of creating a culture that develops Kegan 5 leaders that can take over for the current leaders and help meaningfully change the values as the context changes, in a way that doesn’t simply cause organizations to value drift along with the current broader culture.
How ignoring or not attending for people’s needs creates incentives for motivated reasoning, and how to create spaces that get rid of those incentives WITHOUT being hijacked by whoever screams the loudest.
The importance of cultural tradition and ritual in embedding concepts in augmenting the teaching and telling people what concepts are important.
Can you think of any testable predictions (or retrodictions) that would distinguish my model from your model?
No because I think that our models are compatible. My model is about how to attract, retain, and develop people with high potential or skill that are in alignment your community’s values, and your model says that not retaining, attracting, or developing people that matched our communities values and had high writing skill is what caused it to fail.
If you can give a specific model of why LW1 failed to attract, retain, and develop high quality writers, then I think there’s a better space for comparison. Perhaps you can also point out some testable predictions that each of our models would make.
In contrast, my model has been that communities congregate around predictable sources of high-quality writing, and people who can produce high-quality content in high volume are very rare. Thus, once Eliezer Yudkowsky stopped being active, and Yvain a.k.a. the immortal Scott Alexander moved to Slate Star Codex (in part so that he could write about politics, which we’ve traditionally avoided), all the “intellectual energy” followed Scott to SSC.
First, I want to state that I agree with this model. However, I also want to note that the SSC comments section tend to have fairly low-quality discussion (in comparison to the OB/LW 1.0 heyday), and I’m not sure why this is; candidate hypotheses include that Scott’s explicit politics attracted people with lower epistemic standards, or that the lack of an explicit karma system allowed low-quality discussion to persist (but I don’t think OB had an explicit karma system either?).
Overall, I’m unsure as to what kind of norms/technology maintains high-quality discussion (as opposed to just the presence of discussion in general), and it’s plausible to me that the two may actually be somewhat mutually exclusive (in the sense that norms/technology designed to promote the volume of high-quality discussion may in fact reduce the volume of discussion in general). It’s not clear to me how this tradeoff should be balanced.
in part so that he could write about politics, which we’ve traditionally avoided
I want to state that I agree with this model.
(I sometimes think that I might be well-positioned to fill the market niche that Scott occupied in 2014, but no longer can due to his being extortable (“As I became more careful in my own writings [...]”) in a way that I have been trained not to be. But I would need to learn to write faster.)
One thing is that I think early OBNYC and LW just actually had a lot of chaff comments too. I think people disproportionately remember the great parts.
When you look at the question using that native architecture, it becomes relatively simple to find a reasonable answer. This is the same way that we regularly find solutions to complex negotiations between multiple parties, or plan complex situations with multiple constraints, even though many of those tasks are naively uncomputable.
I’m not confident that it does. I perhaps expect people doing this using the native architecture to feel like they’ve found a reasonable answer. But I would expect them to actually be prioritising their own feelings, in most cases. (Though some people will underweight their own feelings. And perhaps some people will get it right.)
Perhaps they will get close enough for the answer to still count as “reasonable”?
If someone attempts to give equal weight to their own needs, the meds of their interlocutor, and the needs of the forum as a whole—how do we know whether they’ve got a reasonable answer? Does that just have to be left to moderator discretion, or?
If someone attempts to give equal weight to their own needs, the meds of their interlocutor, and the needs of the forum as a whole—how do we know whether they’ve got a reasonable answer? Does that just have to be left to moderator discretion, or?
Yes basically, but if the forum were to take on this direction, the idea would be to have enough case examples/explanations from the moderators about WHY they made that discretion to calibrate people’s reasonable answers. See also this response to Zach which goes more into details about the systems in place to calibrate people’s reasonable answers.
I’m rather confused about what you mean by ‘safe’. I thought I knew what the word meant, but the way you (and some others) are using it perplexes me. Could you explain how to interpret this notion of “safety”?
For instance, this part:
Then there’s the kind that this post is talking about. In this type of environment, it’s safe to say “This conversation is making me feel unsafe, so I need to leave”. It’s also safe to say “It feels like your need for safety is getting in the way of truthseeking” as well as for other people to push back on that if they think that this person’s need for safety is so great in this moment that we need to accommodate them for a bit and return to this topic later.
[emphasis mine]
What do the bolded uses of ‘safe’ mean?
Is it the same meaning as the other uses of ‘safe’ in your comment? If not, what other meanings are in use, in which parts of the comment?
I think it’s best defined by its’ antonym. Unsafety, in this context, would mean anything that triggers a defensive or reactive reaction. Just like how bodily unsafety triggers fear, agression, etc, there are psychological equivalents that trigger the same reaction.
Safety is when a particualr circumstance doesn’t trigger that reaction, OR alternatively there could be a meta safety (AKA, having that reaction doesn’t trigger that reaction, because it’s ok).
I think your bolded definitions of safe would actually be served by changing to the word allowed, which for many people correlates quite closely with their feeling of safety.
I think the question of “what is safety?” is a really good one. I’ll write up some thoughts here both for this thread, but also to be to refer to generally (hence a bit more length).
Safety is when a particular circumstance doesn’t trigger that reaction,
I’m not a fan of that definition. It’s equating “feelings of safety” with “actual safety”
It’s defining safety as the absence the response to perceived unsafety. It feels equivalent to saying “sickness is the thing your immune system fights, and health is the absence of your immune system being triggered to fight something.” Which is very approximately true, but breaks down when you consider autoimmune disorders. With those, it’s the mistaken perception of attack which is the very problem.
This definition can also put a lot of the power in the hands of those who are having a reaction. If we all agree that our conversation must be safe, and that any individual can declare it unsafe because they are having a reaction, this gives a lot power to individuals to force attention on the question of safety (and I fear too asymmetrically with others being blamed for causing the feelings of uncertainty).
-----
So here’s the alternative positive account of “safety” I would give:
One *is* safe if one is unlikely to be harmed; one *feels* if they believe (S1 and/or S2) if they believe they won’t be harmed.
This accords with the standard use of safety, e.g. safety goggles, safety precautions, safe neighborhood, etc.
In conversation, one can be “harmed socially”, e.g. excluded from the group, being “punished” by the group, being made to look bad or stupid (with consequences on how they are treated), having someone act hostilely or aggressive to them (which is a risk of strong negative experience even if they S2 believe it won’t come to any physical or lasting harm), etc. (this is not a carefully developed or complete list).
So in conversation and social spaces, safety equates to not being likely to be harmed in the above ways.
Much the same defenses that activate when feeling under physical threat also come online when feeling under social threat (for indeed, both can be very risky to a human). These are physiological states, fight or flight, etc. How adaptative these are in the modern age . . . more than 0, less than 1 . . .? Having these responses indicates that some part of your mind perceives threat, the question being whether it’s calibrated.
On the question of space: a space can be perceived to have or lower risk of harm to individual (safety) and also higher or lower assessments of risk of harm related to taking specific actions, e.g. saying certain things.
---
With this definition, we can separately evaluate the questions of:
1) Are people actually safe vs likely to be harmed in various ways?
2) Are the harms people worried actually legitimate harms to be worried about?
3) Are people correct to be afraid of being harmed, that is, to feel unsafe?
4) Who should be taking action to cause people to feel unsafe? How is responsibility distributed between the individual and the group?
5) How much should the group/community worry about a) actual safety, and b) perceived safety?
I’m interested in how different people answer these questions generally and in the context of LessWrong.
> I’m not a fan of that definition. It’s equating “feelings of safety” with “actual safety”
I agree with this, but it’s quite a mouthful to deal with. And I think “feelings of safety” are actually more important for truthseeking and creating a product—they’re the things that produce defensiveness, motivated reasoning, etc.
I think mr-hire thinks the important success condition is that people feel safe and that it’s important to design the space towards this goal, with something of a collective responsibility for the feelings of safety of each individual.
This seems rightish- but off in really important ways that I can’t articulate. It’s putting the emphasis on the wrong things and “collective responsiblity” is not an idea I like at all. I think I’d put my stance as something like “feeling unsafe is a major driver of what people say and do, and good cultures provide space to process and deal with those feelings of unsafety”
This definition can also put a lot of the power in the hands of those who are having a reaction. If we all agree that our conversation must be safe, and that any individual can declare it unsafe because they are having a reaction, this gives a lot power to individuals to force attention on the question of safety (and I fear too asymmetrically with others being blamed for causing the feelings of uncertainty).
Note that this issue is explicitly addressed in the original dialogue. If someones feelings are hurting the discourse, they need to take responsibility for that just as much as I need to take responsibility for hurting their feelings. No one is agreeing that all conversations must be safe for all people, but simply that taking into account when people feel unsafe is important.
I agree with this, but it’s quite a mouthful to deal with
Yeah, but there’s a really big difference! You can’t give up that precision.
This seems rightish- but off in really important ways that I can’t articulate.
Nods. Also agree that “collective responsibility” is not the most helpful concept to talk about.
Note that this issue is explicitly addressed in the original dialogue. If someones feelings are hurting the discourse, they need to take responsibility for that just as much as I need to take responsibility for hurting their feelings.
Indeed, the fact people can say “”It feels like your need for safety is getting in the way of truth-seeking”is crucial for it to have any chance.
My expectation based on related real-life experience though, is that if making your need for safety is an option, there will people who abuse this and use it to suck up a lot of time and attention. That technically someone could deny their claim and move on, but this will happen much later than optimal and in the meantime everyone’s attention has been sucked into a great drama. Attempts to say “your safety is disrupting truth-seeking” are accused as being attempts to oppress someone, etc.
This is all imagining how it would go with typical humans. I’m guessing you’re imagining better-than-typical people in your org who won’t have the same failure mode, so maybe it’ll be fine. I’m mostly anchored how I expect that approach to go if applied to most humans I’ve known (especially those really into caring about feelings and who’d be likely to sign up for it).
I think mr-hire thinks the important success condition is that people feel safe and that it’s important to design the space towards this goal, with something of a collective responsibility for the feelings of safety of each individual.
I think Said things that individuals bear full responsibility their feelings of safety, and that it’s actively harmful to make these something the group space has to worry about. I think Said might even believe that “social safety” isn’t even important for the space, i.e., it’s fine if people actually are attacked in social ways, e.g. reputationally harm, caused to be punished by the group, made to experience negative feelings due to aggression from others.
----
If I had to choose between my model of mr-hire’s preferred space and my model of Said’s preferred space, I think I would actually choose Said’s. (Though I might not be correctly characterizing either—I wanted to state my prediction before I asked to test how successfully modeling other’s views).
When it comes to truth seeking, I’d rather err on the side of people getting harmed a bit and having to do a bunch of work to “steel” themselves against the “harsh” environment, then give individuals such a powerful tool (the space being responsible for their perception of being harmed) to disrupt and interfere with discourse. I know that’s not the intended result, but it seems too ripe for abuse to give feelings and needs the primacy I think is being given in the OP scenario. Something like an unachievable utopia: it sounds good, but I am very doubtful it can be done and also be a truth-seeking space.
[Also Said, I had a dream last night that I met you in Central Park, NY. I don’t know what you look or sound like in person, but I enjoyed meeting my dream version of you.]
I think Said things that individuals bear full responsibility their feelings of safety, and that it’s actively harmful to make these something the group space has to worry about.
Well, this is certainly not an egregious strawman by any stretch of the imagination—it’s a reasonable first approximation, really—but I would prefer to be somewhat more precise/nuanced. I would say this:
Individuals bear full responsibility for having their feelings (of safety, yes, and any other relevant propositional attitudes) match the reality as it in fact (objectively/intersubjectively verifiably) presents itself to them.[1]
This, essentially, transforms complaints of “feeling unsafe” into complaints of “being unsafe”; and that is something that we (whoever it is who constitute the “we” in any given case) can consider, and judge. If you’re actually made unsafe by some circumstance, well, maybe we want to do something about that, or prevent it. (Or maybe we don’t, of course. Likely it would depend on the details!) If you’re perfectly safe but you feel unsafe… that’s your own business; deal with it yourself![2]
I think Said might even believe that “social safety” isn’t even important for the space, i.e., it’s fine if people actually are attacked in social ways, e.g. reputationally harm, caused to be punished by the group, made to experience negative feelings due to aggression from others.
The relevant questions, again, are about truth and justice. Is it acceptable for people to be reputationally harmed? Well, how is this happening? Certainly libel is not acceptable. Revealing private information about someone (e.g., about their sexual preferences) is not acceptable. Plenty of other things that might cause reputational harm aren’t acceptable. But if you reveal that I, let us say, falsified scientific data (and if this actually is the case), great reputational harm will be done to me; and this is entirely proper. The fact of the harm itself, in other words, is not dispositive.
Similarly for punishment—punishment is proper if it is just, improper otherwise.
As far as “negative feelings” go… “aggression” is a loaded word; what do you mean by it? Suppose that we are having an in-person debate, and you physically assault me; this is “aggression” that would, no doubt, make me “experience negative feelings”; it would also, obviously, be utterly unacceptable behavior. On the other hand, if you did nothing of the sort, but instead made some cutting remark, in which you subtly impugned my intelligence and good taste—is that “aggression”? Or what if you simply said “Said, you’re completely wrong about this, and mistaken in every particular”… aggression? Or not? I might “experience negative feelings” in each of these cases! But the question of whether any of these behaviors are acceptable, or not, does not hinge primarily on whether they could conceivably be described, in some sense, as “aggression”.
In short… when it comes to deciding what is good and what is bad—as with so many other things—precision is everything.
When it comes to truth seeking, I’d rather err on the side of people getting harmed a bit and having to do a bunch of work to “steel” themselves against the “harsh” environment, then give individuals such a powerful tool (the space being responsible for their perception of being harmed) to disrupt and interfere with discourse. I know that’s not the intended result, but it seems too ripe for abuse to give feelings and needs the primacy I think is being given in the OP scenario.
On this, we entirely agree. (And I would add that it is not simply ripe for abuse; it is, in fact, abused, and rampantly, in all cases I have seen.)
[Also Said, I had a dream last night that I met you in Central Park, NY. I don’t know what you look or sound like in person, but I enjoyed meeting my dream version of you.]
Central Park is certainly a pleasant place to meet anyone! I can only hope that, should we ever meet in fact, I live up to the standards set by my dream self…
“reality as it in fact (objectively/intersubjectively verifiably) presents itself to them”: By this somewhat convoluted turn of phrase I mean simply that it’s conceivable for someone to be deceived—made to perceive the facts erroneously, through adversarial action—in which case it would, obviously, be unreasonable to say that it’s entirely the victim’s responsibility to have their feelings about reality match actual reality instead of reality as they are able to discern it; nevertheless this is not license to say “well, this is what the reality feels like to me”, because “what should you reasonably conclude is the reality, given the facts that, as we can all see, are available to you” is something that may be determined and agreed upon, and in no sense is an individual incorrigible on that question.
Which, of course, does not mean that “what’s the best technique for dealing with feeling unsafe when you’re actually safe” isn’t a topic that the group might discuss.
Thanks for the precise and nuanced write-up, and for not objecting to my crude attempt to characterize your position.
Nothing in your views described here strikes me as gravely mistaken, it seems like a sensible norm set. I suspect that many of our disagreements appear once we attempt be precise around acceptable and not acceptable behaviors and how they are handled.
I agree that “aggression” is fuzzy and that simply causing negative emotions is certainly not the criteria by which to judge the acceptability of behavior. I used those terms to indicate/gesture rather than define.
I have a draft, Three ways to upset people with your speech, which attempts to differentiate between importantly different cases. I find myself looking forward to your comments on it once I finally publish it. I don’t think I would have said that a week ago, and I think it’s largely feeling safer with you, which is in turn the result of greater familiarity (I’ve never been active in the LW comments as much as in the last few weeks). I’m more calibrated about the significance of your words now, the degree of malice behind them (possibly not that much?), and even the defensible positions underlying them. I’ve also updated that it’s possible to have a pleasant and valuable exchange.
(I do not say these things because I wish to malign you with my prior beliefs about you, but because I think they’re useful and relevant information.)
Your warm response to my mentioning dream-meeting you made me feel warm (also learning your Myers Briggs type).
(Okay, now please forgive me for using all the above as part of an “argument”; I mean it all genuinely, but it seems to be a very concrete applied way to discuss topics that have been in the air of late.)
This gets us into some tricky questions I can place in your framework. I think it will take us (+ all the others) a fair bit of conversation to answer, but I’ll mention them here now to at least raise them. (Possibly just saying this because I’m away this week and plan not to be online much.)
My updates on you (if correct) suggest that largely Said’s comments do not threaten me much and I shouldn’t feel negative feelings as a result. Much of this is just how Said talks, and he’s still interested in honest debate, not just shutting you down with hostile talk. But my question is the “reality as it presents itself to me” you mentioned. The reality might be that Said is safe, but was I, given my priors and evidence available to me before, wrong to be afraid before I gained more information about how to interpret Said?
(Maybe I was, but this is not obvious.)
Is the acceptability of behavior determined by what the recipient reasonably could have believed (as judged by . . . ?) or by the actual reality. Or there are three possibilities even: 1) what I could have reasonably believed was the significance of your actions, 2) what you could have reasonably believed was the significance of your actions, 3) what the actual significance of your actions were (if this can even be defined sensibly).
It does seem somewhat unfair if the acceptability of your behavior is impacted by what I can reasonably believe. It also seems somewhat unfair that I should experience attack because I reasonably lacked information.
How do we handle all this? I don’t definitively know. Judging what is acceptable/reasonable/fair and how all different perspectives add up . . . it’s a mess that I don’t think gets better even with more attempt at precision. I mostly want to avoid having to judge.
This is in large part what intuitively pushes me towards wanting people to be proactive in avoiding misinterpretations and miscalibrations of other’s intent—so we don’t have to judge who was at fault. I want people to give people enough info that they correctly know even when I’m harsh, I still want them to feel safe. Mostly applies to people who don’t know me well. Once the evidence has accrued and you’re calibrated on what things mean, you require little “padding” (this is my version of Combat culture essentially), but you’ve got to accrue that evidence and establish the significance of actions with others first.
--
Phew, like everything else, that was longer than expected. I should really start expected everything to be long.
Curious if this provides any more clarity on my position (even if it’s not persuasive) and curious where you disagree with this treatment.
I ended up having thoughts here that grew beyond the context (how to think about this feels related to how to think about depleted willpower). Wrote a shortform post.
My current best guess is that you’d get the best results (measured roughly “useful ideas generated and non-useful ideas pruned”) from a collection of norms where
a) people need to take responsibility for their own feelings of fear, and there is help/guidelines on how to go about that if you’re not very good at it yet, and
b) people also take responsibility for learning to the social and writing skills to avoid particularly obvious failure modes.
i.e. “I’m feeling defensive” shouldn’t be a get out of jail free card. (in particular, any request that someone change their communication behavior should come with a corresponding costly signal that you are working on improving your ability to listen while triggered)
And while I think I’ve believed this for a couple weeks, I don’t think I was doing the work to actually embody it, and I think that’s been a mistake I’ve been making.
I’ve been trying to use the phrase ‘feeling of safety’ when it comes up but it has the unfortunate property that ‘aspiring rationalist’ had, where there isn’t a stable equilibrium where people reliably say the whole phrase.
Over the past couple years, I have updated to “yes, LessWrong should be the place focused on truthseeking.”
Updated to? This wording surprises me, because I’m having trouble forming a hypothesis as to what your earlier position could have been. (I’m afraid I haven’t studied your blogging corpus.) What else is this website for, exactly?
My steelman of this position is something like, “I favored focusing instrumental rationality because it seemed, well, useful. At the time I figured that this was just a different subject than epistemic rationality, & focusing on it would at worst mean less progress improving the accuracy of our beliefs. But in hindsight this involved allowing epistemics to get worse for the sake of more instrumental success. I’ve now updated towards that having been a bad tradeoff.”
Thanks! I’m not sure this is a place where steelmanning is quite the appropriate tool. My past self was optimized for being my past self, not being right. He was mostly just not trying to solve this question.
But, in this case, I think the best tool is more properly called “modeling people” and maybe “empathy”.
Things my past self cared about and/or believed included:
All the probability stuff feels too hard to think about, and it doesn’t seem like it’s really going to help me that much even if I put a lot of work into it. So for me personally, I’m just going to try to “remember base rates” and a few other simple heuristics and call it a day. I was glad other people took it more seriously though
Truth seems like one of many important things. What matters is getting things accomplished. (I’ve never been optimizing against truth, I have just prioritized other things. There’s been times where I, say, only put 20 minutes into checking an essay for being right, rather than 2 hours, when I had reason to suspect I might have had motivated reasoning.)
I thought (and still think, although less strongly and for more nuanced reasons) that the in person rationality community is unhealthy because it only selects for a few narrow types of person, who are min-maxed in a particular skillset. And I think the in person community is important (both for epistemic and instrumental reasons). It is important to be a community that doesn’t actively drive away people who bring other skills to the table.
I still roughly believe all that. The main update is that there should a) be dedicated spaces that focus on truthseeking as their [probably] sacred value, b) that LessWrong should be such a space. (But, as noted in Tensions in Truthseeking, there are still different tradeoffs you can make in your truthseeking frame, and I think it’s good to have spaces that have made different min-max tradeoffs to explore those tradeoffs. For example, there might be math-heavy spaces, there might be “blunt communication” spaces that optimize for directness, there might be feelings-heavy spaces that optimize for understanding and owning your internal state)
(I have made a bit of conceptual progress on probability stuff. I probably will never do real Bayesian Wizardry but I think grok it better now – I can follow some conversations I didn’t used to be able to follow and in some cases I can participate in and uphold norms that help others on their way to learning it better than I)
There is an interesting thing in all this space I recently re-read while perusing the old critiques of Gleb. A paraphrase of the linked comment is:
I think a problem with effective altruists is they often end up with a conception that marketing is icky, and that without marketing they are ineffective. I think Gleb might have just said “I’d rather be effective and icky than ineffective and pure.” And this is maybe an unhelpful frame that other people are implicitly using. There are ways you can market effectively without actually being icky.
And, while I’m not sure, I think I might have held a frame somewhat like that (I don’t have clear memories of biting either particular bullet). But my current position is “effective altruists should hold to a high epistemic standard, even when marketing. But, learn to market well within those constraints.”
Okay, but I thought the idea was that instrumental rationality and epistemic rationality are very closely related. Two sides of the same coin, not two flavors of good thing that sometimes trade off against each other. That agents achieve their goals by means of building accurate models, and using those models to “search out paths through probability” that steer the world into the desired goal-state. If the models aren’t accurate, the instrumental probability-bending magic doesn’t work and cannot work.
Okay, but geez man, my past self had different beliefs. What do you want here? What is your incredulity here aiming to accomplish? If you can’t simulate the mind of a person who showed up on LessWrong with one set of beliefs and gradually updated their beliefs in a set of directions that are common on the site, I think you should prioritize learning to simulate other minds a bit
What is your incredulity here aiming to accomplish?
I genuinely feel incredulous and am trying to express what I’m actually thinking in clear language? I mean, it’s also totally going to be the case that the underlying generator of “genuinely felt incredulity” is no doubt going to be some sort of subconscious monkey-politics status move designed by evolution to make myself look good at the expense of others. It’s important to notice that! But the mere fact of having noticed that doesn’t make the feeling go away, and given that the feeling is there, it’s probably going to leak into my writing. I could expend more effort doing a complicated System-2 political calculation that tries to simulate you and strategically compute what words I should say in order to have the desired effect on you. But not only is that more work than saying what I’m actually thinking in clear language, I also expect it to result in worse writing. Use the native architecture!
I mean, if it’ll help, we can construct a narrative in which my emotion of incredulity that was designed by evolution to make me look good, actually makes me look bad in local social reality? That’s a win-win Pareto improvement: I don’t have to mutilate my natural writing style in the name of so-called “cooperative” norms, and you don’t have to let my monkey-politics brain get away with “winning” the interaction.
How about this? Incredulity is, definitionally, a failed prediction. The fact that I felt incredulous means that my monkey status instincts are systematically distorting my anticipations about the world, making me delusionally perceive things as “obvious” exactly when they’re things that I coincidentally happened to already know, and not because of their actual degree-of-obviousness as operationalized by what fraction of others know them. (And conversely, I’ll delusionally perceive things as “nonobvious” exactly when I coincidentally happened to not-know them.)
(Slaps forehead)Hello, Megan! Ten years into this “rationality” business, and here I am still making rookie mistakes like this! How dumb can I get?
I think you should prioritize learning to simulate other minds a bit
Thanks, this is a good suggestion! I probably am below average at avoiding the typical mind fallacy. You should totally feel superior to me on this account!
I think there are separate worthwhile skills of “focus on learning empathy/modeling and let clear language flow from that”, and also “writing skills exist that are separate from epistemics” (such as brevity, which I think actually factors in here a bit)
Something that may not have been clear from my past discussion is that when I say “this could have been written in a way that was less triggering”, or something, I’m not (usually) meaning that to be a harsh criticism. Just, the sort of thing that you should say ‘ah, that makes sense. I will work on that’ for the future.
Just, the sort of thing that you should say ‘ah, that makes sense. I will work on that’ for the future.
It’s actually not clear to me that I should work on that. As a professional hazard of my other career, I’m pretty used to people trying to use “You would be more persuasive if you were nicer” as an attempted silencing tactic; if I just believed everyone who told me that, I would never get anything done.
Only the latter. And also the vehemence with which these viewpoints seemed to be held and defended. I got the impression that statements of the sort “yay truth as the only sacred value” received strong support; personally I find that off-putting in many contexts.
Edit: The reason I find it off-putting isn’t that I disagree with the position as site policy. More that sometimes the appropriate thing in a situation isn’t just to respond with some tirade about why it’s good to have an unempathetic site policy.
To give some more context: Only the first instance of this had to do with explicit calls for forum policy. This was probably the same example that inspired the dialogue between Jill and John above.
The second example was a comment on the question of making downvotes less salient. While I agree that the idea has drawbacks, I was a bit perplexed that a comment arguing against it got strongly upvoted despite including claims that felt to me like problematic “rationality for rationality’s sake”: Instead of allowing people to only look at demotivating information at specific times, we declare it antithetical to the “core of rationality” to hide information whether or not it overall makes people accomplish their goals better.
The third instance was an exchange you had about conversational tone and (lack of) charity. Toward the end you said that you didn’t like the way you phrased your initial criticism, but my quick impression (and I probably only skimmed the lengthy exchange and also don’t remember details) was that I generally thought your points seemed pretty defensible, and the way your conversation partner commented would have also thrown me off. “Tone and degree of charity are very important too” is a perspective I’d like to see represented more among LW users. (But if I’m in the minority, that’s fine and I don’t object to communities keeping their defining features if the majority feels that they are benefitting.)
Maybe I expressed it poorly, but what I meant was just that rationality is not an end in itself. If I complain that some piece of advice is not working for me because it makes me (all-things-considered, long-term) less productive (towards the things that are most important to me) and less happy, and my conversation partner makes some unqualified statement to the degree of “but it’s rational to follow this type of advice”, I will start to suspect that they are misunderstanding what rationality is for.
I agree there’s something like vehemence and it’s made all the conversations unpleasant and stressful. Someone countered to me that if you perceive someone to be threatening the very integrity of your ability to have conversations, it’s appropriate to break frame and get up in arms. I’m not convinced it’s warranted here, but maybe...
I’m not sure about the exact proportion of people’s perspectives. There definitely is a cluster of people (myself included) who think “tone”, etc. are significant. (This group also might be more averse to getting into online conflicts.) I’m also concerned about the number of people who would counterfactually engage more on LessWrong, except they dislike the conversations they’ll end up in currently.
There are a bunch of conversations going on about the topic right (some in semi-private which might be public soonish). There’s support (at least on the LW team) for an Archipelago type solution where people can opt-in into one of 2 or 3 norm sets. (Though that doesn’t quite fix site-level things like the karma notifier settings.) One of those spaces should have much more “civility.”
Yeah, that’s reasonable. I think that many people, while agreeing with that (or something close to it), get very afraid as soon as someone says it that because they fear it’s going to be used to justify distinctly not-rational/damages the whole endeavor of being rational. I have some of this fear myself.
It seems to me that rationality is extremely fragile and vulnerable, such that even though rationality might serves other goals, you have to be very uncompromising with regards to rationality, especially core things like hiding information from yourself (I was lightly opposed to the negative karma hiding myself) even if it that has appararant costs.
But it’s hard. I think there are tricky questions to answer, but the conversation currently can be civil/happen without vehemence.
Cool! And I appreciate the difficulty of the task at hand. :)
When I model these conversations, one failure mode I’m worried about is that the “more civility” position gets lumped together with other things that Lesswrong is probably right to be scared of.
So, the following is to delineate my own views from things I’m not saying:
I could imagine being fine with Bridgewater culture in many (but not all) contexts. I hate that in “today’s climate” it is difficult to talk about certain topics. I think it’s often the case that people complaining about tone or about not feeling welcome shouldn’t expect to have their needs accommodated.
And yet I still find some features of what I perceive to be “rationalist culture” very off-putting.
I don’t think I phrased it as well in my first comment, but I can fully get behind what Raemon said elsewhere in this thread:
So it’s not that I’m saying that I’d prefer a culture where truth-seeking is occasionally completely abandoned because of some other consideration. Just that the side that superficially looks more virtuous when it comes to truth-seeking (for instance because they boldly proclaim the importance of not being bothered by tone/tact, downvote notifications, etc.) isn’t automatically what’s best in the long run.
Edited to add: I admit it’s a delicate balance to walk. But sometimes, people are inconsiderate in a way that definitely harms discussions. The principle of charity isn’t just a thing in philosophy to make people feel good; there’s also some methodological use to it. Likewise with trying to understand that other people have different minds from one’s own. There has to be a way to point out inconsiderateness that doesn’t get met with a response a la “tact doesn’t matter because truth is the only virtue.”
I agree with that. But people can have very different psychologies. Most people are prone to overconfidence, but some people are underconfident and beat themselves up too much over negative feedback. If the site offers an optional feature that is very useful for people of the latter type, it’s at least worth considering whether that’s an overall improvement. I wasn’t even annoyed that people didn’t like the feature; it was more about the way in which the person argued. Generally, more display of awareness of people having different psychologies would please me. :)
I also find it off-putting in many contexts—perhaps most contexts. But if there’s any consequentialist value in having one space in the entire world where (within the confines of that space) truth is the only sacred value, perhaps lesswrong.com is a Schelling point?
Something that I’m maybe able to put into words now:
The classical example of “sacred values run amok” in my mind is when you ask people how much money a hospital should spend on a heart transplant for a dying child. People try to dodge the question, avoiding trading off a sacred value for a mundane value. Despite the fact that money can buy hospital equipment that saves other lives.
It’s plausible that hospital should hold “keeping people healthy and alive” as an overall sacred value, which they never trade off against. This might forbid some paths where resources are spent on things that weren’t necessary to keep people healthy and alive. But it doesn’t tell you what are the best strategies to go about it are. You’re allowed to sacrifice a boy’s life to buy hospital equipment. You’re even allowed to sacrifice a boy’s life to make sure your employees are well rested and not overly stressed. Running a hospital is a marathon, not a sprint.
Over the past couple years, I have updated to “yes, LessWrong should be the place focused on truthseeking.” I think I came to believe that right around the time I wrote Tensions in Truthseeking, in the process of writing the paragraph about instrumental sacredness. But that tells us what the question is, not what the answer is.
Some of the language about “holding truth sacred” (things you’ve said, and others) has came across to me with a tone of single-minded focus that feels like not being willing to put an upper bound on a heart transplant, rather than earnestly asking the question “how do we get the most valuable truthseeking the most effective way?”
There’s also the bit where operationalization matters. “Minimize falsehood” is a different function than “maximize true, good ideas over time” which is a different function than “maximize true, good ideas that are communicated well enough to impact the world.”
I definitely agree that there could exist perverse situations where there are instrumental tradeoffs to be made in truthseeking of the kind I and others have been suspicious of. For lack of a better term, let me call these “instrumentally epistemic” arguments: claims of the form, “X is true, but the consequences of saying it will actually result in less knowledge on net.” I can totally believe that some instrumentally epistemic arguments might hold. There’s nothing in my understanding of how the universe works that would prevent that kind of scenario from happening.
But in practice, with humans, I expect that a solid supermajority of real-world attempts to explicitly advocate for norm changes on “instrumentally epistemic” grounds are going to be utterly facile rationalizations with the (typically unconscious) motivation of justifying cowardice, intellectual dishonesty, ego-protection, &c.
I (somewhat apologetically) made an “instrumentally epistemic” argument in a private email thread recently, and Ben seemed super pissed in his reply (bold italics, incredulous tone, ”?!?!?!?!?!” punctuation). But the thing is—even if I might conceivably go on to defend a modified form of my original argument—I can’t blame Ben for using a pissed-off tone in his reply. “Instrumentally epistemic” arguments are an enormous red flag—an infrared flag thirty meters wide. Your prior should be that someone making an “instrumentally epistemic” argument can be usefully modeled as trying to undermine your perception of reality and metaphorically slash your tires (even if their conscious phonological loop never contains the explicit sentence “And now I’m going to try to undermine Ray Arnold’s perception of reality”).
Now, maybe that prior can be overcome for some arguments and some arguers! But the apparent failure of the one making the “instrumentally epistemic” argument to notice the thirty-meter red flag, is another red flag.
I don’t think the hospital example does the situation justice. The trade-off of choosing whether to spend money on a heart transplant or nurse salaries doesn’t seem analogous to choosing between truth and the occasional allegedly-instrumentally-epistemic lie (like reassuring your interlocutor that you respect them even when you don’t, in fact, respect them). Rather, it seems more closely analogous to choice of inquiry area (like whether to study truths about chemistry, or truths about biology), with “minutes of study time” as the resource to be allocated rather than dollars.
If we want a maximally charitable medical analogy for “instrumentally epistemic” lies, I would instead nominate chemotherapy, where we deliberately poison patients in the hope of hurting cancer cells more than healthy cells. Chemotherapy can be good if there’s solid evidence that you have a specific type of cancer that responds well to that specific type of chemotherapy. But you should probably check that people aren’t just trying to poison you!
I’m not advocating lying here, I’m advocating learning the communication skills necessary to a) actually get people to understand your point (which they’ll have a harder time with if they’re defensive), and b) not wasting dozens of hours unnecessarily (which could be better spent on figuring other things out).
[and to be clear, I also advocate gaining the courage to speak the truth even if your voice trembles, and be willing to fight for it when it’s important. Just, those aren’t the only skills a rationalist or a rationalist space needs. Listening, communicating clearly, avoiding triggering people’s “use language as politics mode”, and modeling minds and frames different from your own are key skills too]
Why do you think this prior is right?
This seems true only if your prior is so obviously right that one couldn’t disagree with it in good faith. I’m not convinced of this.
(As I mentioned I’m sympathetic to both sides of the debate here, but I find myself wanting to question your side more, because it seems to display a lot more certainty (along with associated signals such as exasperation and incredulity), which doesn’t seem justified to me.)
Thanks, I appreciate it a lot! You should be questioning my “side” as harshly as you see fit, because if you ask questions I can’t satisfactorily answer, then maybe my side is wrong, and I should be informed of this in order to become less wrong.
The mechanism by which saying true things leads to more knowledge is at least straightforward: you present arguments and evidence, and other people evaluate those arguments and evidence using the same general rules of reasoning that they use for everything else, and hopefully they learn stuff.
In order for saying true things to lead to less knowledge, we need to postulate some more complicated failure mode where some side-effect of speech disrupts the ordinary process of learning. I can totally believe that such failure modes exist, and even that they’re common. But lately I seem to be seeing a lot of arguments of the form, “Ah, but we need to coordinate in order to create norms that make everyone feel Safe, and only then can we seek truth.” And I just … really have trouble taking this seriously as a good faith argument rather than an attempt to collude to protect everyone’s feelings? Like, telling the truth is not a coordination problem? You can just unilaterally tell the truth.
Hm, I think there’s a risk of signal miscalibration here. Just because I feel exasperated and this emotion leaks into my writing, doesn’t necessarily mean implied probabilities close to 1? (Related: Say It Loud. See also my speculative just-so story about why the incredulity is probably non-normative.)
(It’s 1:20 a.m. on Sunday and I’ve used up my internet quota for the weekend, so it might take me a few days to respond to future comments.)
I want to address something that I think is quite important in the context of this post, because I think you’re pattern matching the “let’s make a space where people’s needs are addressed,” to the standard social justice safe space, but there are actually 3 types of safe spaces, and the one you’re imagining is not related to the ones this post is talking about.
The social justice kind, where nobody is allowed to bring up arguments that make you feel unsafe, is the one you’re talking about. “We need to make everyone feel safe and can’t seek truth until we do that” is describing an environment where truth seeking is basically impossible. I think private spaces like that are important in a rationalist environment, because some people are fragile and need to heal before they can participate in truth seeking, but are almost never right for an organization that has the goal of seeking truth.
Then there’s the kind that this post is talking about. In this type of environment, it’s safe to say “This conversation is making me feel unsafe, so I need to leave”. It’s also safe to say “It feels like your need for safety is getting in the way of truthseeking” as well as for other people to push back on that if they think that this person’s need for safety is so great in this moment that we need to accommodate them for a bit and return to this topic later. I think the majority of public truth-seeking spaces would be served by adopting this type of safety, in lieu of something like Crocker’s rules.
Then there’s the third type of safe space. In this type of safe space, you can say “This topic is making me feel unsafe” and the expected response is “Awesome, then we’re going to keep throwing you in as many situations like this as possible, poke that emotional wound, and help you work through it so you can level up as an individual and we can level up as an organization.” In this case, the safety comes from the strict vetting procedures and strong culture that let you know that people poking your are sincere and skilled, and the people being poked have the emotional strength to deal with it. I think that a good majority of PRIVATE truth seeking spaces should strive to be this third type of safe space.
One of the mistakes I made in this post was conflate the second and third types of safe spaces, so for instance I posited a public space that also had radical transparency, which is really only a tool you should use in a culture with strong vetting. However, I definitely was not suggesting the first type of safe space, but I get the impression that that’s what you keep imagining.
I mean, in the case of a website that people use in their free time, you don’t necessarily even need an excuse: if you don’t find a conversation valuable (because it’s making you feel unsafe or for any other reason), you can just strong-downvote them and stop replying.
There was a recent case on Less Wrong where one of two reasons I gave for calling for end-of-conversation was that I was feeling “emotionally exhausted”, which seems similar to feeling unsafe. But that was me explaining why I didn’t feel like talking anymore. I definitely wasn’t saying that my interlocutor should give equal weight to his needs, my needs, and the needs of the forum of the whole. I don’t see how anyone is supposed to compute that.
If your primary metaphor for thought is simple computations or mathematical functions, I can see how this would be very confusing, but I don’t think that’s actually the native architecture of our brains. Instead our brain is noticing patterns, creating reusable heuristics, and simulating other people using empathy.
When you look at the question using that native architecture, it becomes relatively simple to find a reasonable answer. This is the same way that we regularly find solutions to complex negotiations between multiple parties, or plan complex situations with multiple constraints, even though many of those tasks are naively uncomputable. The shared values and culture serve to make sure those heuristics are calibrated similarly between people. Reply
I don’t think “reasonable” is the correct word here. You keep assuming away the possibility of conflict. It’s easy to find a peaceful answer by simulating other people using empathy, if there’s nothing anyone cares about more than not rocking the boat. But what about the least convenient possible world where one party has Something to Protect which the other party doesn’t think is “reasonable”?
Riiiight, about that. The OP is about robust organizations in general without mentioning any specific organization, but given the three mentions of “truthseeking”, I’d like to talk about the special case of this website, and set it in the context of a previous discussion we’ve had.
I don’t think the OP is compatible with the shared values and culture established in Sequences-era Overcoming Bias and Less Wrong. I was there (first comment December 22, 2007). If the Less Wrong and “rationalist” brand names are now largely being held by a different culture with different values, I and the forces I represent have an interest in fighting to take them back.
Let me reply to your dialogue with another. To set the scene, I’ve been drafting a forthcoming post (working title: “Schelling Categories, and Simple Membership Tests”) in my nascent Sequence on the cognitive function of categories, which is to refer back to my post “The Univariate Fallacy”. Let’s suppose that by the time I finally get around to publishing “Schelling Categories” (like the Great Teacher, I suffer from writer’s molasses), the Jill from your dialogue has broken out of her simulation, instantiated herself in our universe, and joined the LW2 moderation team.
Jill: Zack, I’ve had another complaint—separate from the one in May—about your tendency to steer conversations towards divisive topics, and I’m going to ask you to tone it down a bit when on Frontpage posts.
Zack: What? Why? Wait, sorry—that was a rhetorical question, which I’ve been told is a violation of cooperative discourse norms. I think I can guess what motivated the complaint. But I want to hear you explain it.
Jill: Well, you mentioned this “univariate fallacy” again, and in the context of some things you’ve Tweeted, there was some concern that you were actually trying to allude to gender differences, which might make some community members of marginalized genders feel uncomfortable.
Zack: (aside) I’m guess I’m glad I didn’t keep calling it Lewontin’s fallacy.
(to Jill) So … you’re asking me to tone down the statistics blogging—on less wrong dot com—because some people who read what I write elsewhere can correctly infer that my motivation for thinking about this particular statistical phenomenon was because I needed it to help me make sense of an area of science I’ve been horrifiedly fascinated with for the last fourteen years, and that scientific question might make some people feel uncomfortable?
Jill: Right. Truthseeking is very important. However, it’s clear that just choosing one value as sacred and not allowing for tradeoffs can lead to very dysfunctional belief systems. I believe you’ve pointed at a clear tension in our values as they’re currently stated: the tension between freedom of speech and truth, and the value of making a space that people actually want to have intellectual discussions at. I’m only asking you to give equal weight to your own needs, the needs of the people you’re interacting with, and the needs of the organization as a whole.
Zack: (aside) Wow. It’s like I’m actually living in Atlas Shrugged, just like Michael Vassar said. (to Jill) No.
Jill: What?
Zack: I said No. As a commenter on lesswrong.com, my duty and my only duty is to try to make—wait, scratch the “try”—to make contributions that advance the art of human rationality. I consider myself to have a moral responsibility to ignore the emotional needs of other commenters—and symmetrically, I think they have a moral responsibility to ignore mine.
Jill: I’d prefer that you be more charitable and work to steelman what I said.
Zack: If you think I’ve misunderstood what you’ve said, I’m happy to listen to you clarify whatever part you think I’m getting wrong. The point of the principle of charity is that people are motivated to strawman their interlocutors; reminding yourself to be “charitable” to others helps to correct for this bias. But to tell others to be charitable to you without giving them feedback about how, specifically, you think they’re misinterpreting what you said—that doesn’t make any sense; it’s like you’re just trying to mash an “Agree with me” button. I can’t say anything about what your conscious intent might be, but I don’t know how to model this behavior as being in good faith—and I feel the same way about this new complaint against me.
Jill: Contextualizing norms are valid rationality norms!
Zack: If by “contextualizing norms” you simply mean that what a speaker means needs to be partially understood from context, and is more than just what the sentence the speaker said means, then I agree—that’s just
former Denver Broncos quarterback Brian Griesephilosopher of language H. P. Grice’s theory of conversational implicature. But when I apply contextualizing norms to itself and look at the context around which “contextualizing norms” was coined, it sure looks like the entire point of the concept is to shut down ideologically inconvenient areas of inquiry. It’s certainly understandable. As far as the unwashed masses are concerned, it’s probably for the best. But it’s not what this website is about—and it’s not what I’m about. Not anymore. I am an aspiring epistemic rationalist. I don’t negotiate with emotional blackmailers, I don’t double-crux with Suicide Rock, and I’ve got Something to Protect.Jill: (baffled) What could possibly incentivize you to be so unpragmatic?
Zack: It’s not the incentives! (aside) It’s me!
(Curtain.)
Yes, if someone has values that are in fact incompatible with the culture of the organization, they shouldn’t be joining that organization. I thought that was clear in my previous statements, but it may in fact have not been. If every damn time their own values are at odds with what are best for the organization given its’ values, that’s an incompatible difference. They should either find a different organization, or try the archipeligo model. There are such thing as irreconcilable value differences.
I agree. I think when that culture was established, the community was missing important concepts about motivated reasoning and truth seeking and chose values that were in fact not optimized for the ultimate goal of creating a community that could solve important problems.
I think it is in fact good to experiment with the norms you’re talking about from the original site, but I think many of those norms originally caused the site to decline and people to go elsewhere. Given my current mental models, I predict a site that uses those norms to make less intellectual progress than a similar site using my norms although I expect you to have the opposite intuition. As I stated in the introduction, the goal of this post was simply to make sure that those mental models were in discourse.
Re your dialogue: The main thing that I got from it was that you think a lot of the arguments in the OP are motivated reasoning and will lead to bad incentives. I also got that this is a subject you care a lot about.
Can you be more specific? Can you name three specific concepts about motivated reasoning and truthseeking that you know, but Sequences-era Overcoming Bias/Less Wrong didn’t?
I mean, that’s one hypothesis. In contrast, my model has been that communities congregate around predictable sources of high-quality writing, and people who can produce high-quality content in high volume are very rare. Thus, once Eliezer Yudkowsky stopped being active, and Yvain a.k.a. the immortal Scott Alexander moved to Slate Star Codex (in part so that he could write about politics, which we’ve traditionally avoided), all the “intellectual energy” followed Scott to SSC.
Can you think of any testable predictions (or retrodictions) that would distinguish my model from your model?
Yes. Thanks for listening.
Here are a few:
The importance of creating a culture that develops Kegan 5 leaders that can take over for the current leaders and help meaningfully change the values as the context changes, in a way that doesn’t simply cause organizations to value drift along with the current broader culture.
How ignoring or not attending for people’s needs creates incentives for motivated reasoning, and how to create spaces that get rid of those incentives WITHOUT being hijacked by whoever screams the loudest.
The importance of cultural tradition and ritual in embedding concepts in augmenting the teaching and telling people what concepts are important.
No because I think that our models are compatible. My model is about how to attract, retain, and develop people with high potential or skill that are in alignment your community’s values, and your model says that not retaining, attracting, or developing people that matched our communities values and had high writing skill is what caused it to fail.
If you can give a specific model of why LW1 failed to attract, retain, and develop high quality writers, then I think there’s a better space for comparison. Perhaps you can also point out some testable predictions that each of our models would make.
First, I want to state that I agree with this model. However, I also want to note that the SSC comments section tend to have fairly low-quality discussion (in comparison to the OB/LW 1.0 heyday), and I’m not sure why this is; candidate hypotheses include that Scott’s explicit politics attracted people with lower epistemic standards, or that the lack of an explicit karma system allowed low-quality discussion to persist (but I don’t think OB had an explicit karma system either?).
Overall, I’m unsure as to what kind of norms/technology maintains high-quality discussion (as opposed to just the presence of discussion in general), and it’s plausible to me that the two may actually be somewhat mutually exclusive (in the sense that norms/technology designed to promote the volume of high-quality discussion may in fact reduce the volume of discussion in general). It’s not clear to me how this tradeoff should be balanced.
(I sometimes think that I might be well-positioned to fill the market niche that Scott occupied in 2014, but no longer can due to his being extortable (“As I became more careful in my own writings [...]”) in a way that I have been trained not to be. But I would need to learn to write faster.)
One thing is that I think early OBNYC and LW just actually had a lot of chaff comments too. I think people disproportionately remember the great parts.
I’m not confident that it does. I perhaps expect people doing this using the native architecture to feel like they’ve found a reasonable answer. But I would expect them to actually be prioritising their own feelings, in most cases. (Though some people will underweight their own feelings. And perhaps some people will get it right.)
Perhaps they will get close enough for the answer to still count as “reasonable”?
If someone attempts to give equal weight to their own needs, the meds of their interlocutor, and the needs of the forum as a whole—how do we know whether they’ve got a reasonable answer? Does that just have to be left to moderator discretion, or?
Yes basically, but if the forum were to take on this direction, the idea would be to have enough case examples/explanations from the moderators about WHY they made that discretion to calibrate people’s reasonable answers. See also this response to Zach which goes more into details about the systems in place to calibrate people’s reasonable answers.
I’m rather confused about what you mean by ‘safe’. I thought I knew what the word meant, but the way you (and some others) are using it perplexes me. Could you explain how to interpret this notion of “safety”?
For instance, this part:
[emphasis mine]
What do the bolded uses of ‘safe’ mean?
Is it the same meaning as the other uses of ‘safe’ in your comment? If not, what other meanings are in use, in which parts of the comment?
I think it’s best defined by its’ antonym. Unsafety, in this context, would mean anything that triggers a defensive or reactive reaction. Just like how bodily unsafety triggers fear, agression, etc, there are psychological equivalents that trigger the same reaction.
Safety is when a particualr circumstance doesn’t trigger that reaction, OR alternatively there could be a meta safety (AKA, having that reaction doesn’t trigger that reaction, because it’s ok).
I think your bolded definitions of safe would actually be served by changing to the word allowed, which for many people correlates quite closely with their feeling of safety.
I think the question of “what is safety?” is a really good one. I’ll write up some thoughts here both for this thread, but also to be to refer to generally (hence a bit more length).
I’m not a fan of that definition. It’s equating “feelings of safety” with “actual safety”
It’s defining safety as the absence the response to perceived unsafety. It feels equivalent to saying “sickness is the thing your immune system fights, and health is the absence of your immune system being triggered to fight something.” Which is very approximately true, but breaks down when you consider autoimmune disorders. With those, it’s the mistaken perception of attack which is the very problem.
This definition can also put a lot of the power in the hands of those who are having a reaction. If we all agree that our conversation must be safe, and that any individual can declare it unsafe because they are having a reaction, this gives a lot power to individuals to force attention on the question of safety (and I fear too asymmetrically with others being blamed for causing the feelings of uncertainty).
-----
So here’s the alternative positive account of “safety” I would give:
One *is* safe if one is unlikely to be harmed; one *feels* if they believe (S1 and/or S2) if they believe they won’t be harmed.
This accords with the standard use of safety, e.g. safety goggles, safety precautions, safe neighborhood, etc.
In conversation, one can be “harmed socially”, e.g. excluded from the group, being “punished” by the group, being made to look bad or stupid (with consequences on how they are treated), having someone act hostilely or aggressive to them (which is a risk of strong negative experience even if they S2 believe it won’t come to any physical or lasting harm), etc. (this is not a carefully developed or complete list).
So in conversation and social spaces, safety equates to not being likely to be harmed in the above ways.
Much the same defenses that activate when feeling under physical threat also come online when feeling under social threat (for indeed, both can be very risky to a human). These are physiological states, fight or flight, etc. How adaptative these are in the modern age . . . more than 0, less than 1 . . .? Having these responses indicates that some part of your mind perceives threat, the question being whether it’s calibrated.
On the question of space: a space can be perceived to have or lower risk of harm to individual (safety) and also higher or lower assessments of risk of harm related to taking specific actions, e.g. saying certain things.
---
With this definition, we can separately evaluate the questions of:
1) Are people actually safe vs likely to be harmed in various ways?
2) Are the harms people worried actually legitimate harms to be worried about?
3) Are people correct to be afraid of being harmed, that is, to feel unsafe?
4) Who should be taking action to cause people to feel unsafe? How is responsibility distributed between the individual and the group?
5) How much should the group/community worry about a) actual safety, and b) perceived safety?
I’m interested in how different people answer these questions generally and in the context of LessWrong.
> I’m not a fan of that definition. It’s equating “feelings of safety” with “actual safety”
I agree with this, but it’s quite a mouthful to deal with. And I think “feelings of safety” are actually more important for truthseeking and creating a product—they’re the things that produce defensiveness, motivated reasoning, etc.
This seems rightish- but off in really important ways that I can’t articulate. It’s putting the emphasis on the wrong things and “collective responsiblity” is not an idea I like at all. I think I’d put my stance as something like “feeling unsafe is a major driver of what people say and do, and good cultures provide space to process and deal with those feelings of unsafety”
Note that this issue is explicitly addressed in the original dialogue. If someones feelings are hurting the discourse, they need to take responsibility for that just as much as I need to take responsibility for hurting their feelings. No one is agreeing that all conversations must be safe for all people, but simply that taking into account when people feel unsafe is important.
Yeah, but there’s a really big difference! You can’t give up that precision.
Nods. Also agree that “collective responsibility” is not the most helpful concept to talk about.
Indeed, the fact people can say “”It feels like your need for safety is getting in the way of truth-seeking”is crucial for it to have any chance.
My expectation based on related real-life experience though, is that if making your need for safety is an option, there will people who abuse this and use it to suck up a lot of time and attention. That technically someone could deny their claim and move on, but this will happen much later than optimal and in the meantime everyone’s attention has been sucked into a great drama. Attempts to say “your safety is disrupting truth-seeking” are accused as being attempts to oppress someone, etc.
This is all imagining how it would go with typical humans. I’m guessing you’re imagining better-than-typical people in your org who won’t have the same failure mode, so maybe it’ll be fine. I’m mostly anchored how I expect that approach to go if applied to most humans I’ve known (especially those really into caring about feelings and who’d be likely to sign up for it).
I think mr-hire thinks the important success condition is that people feel safe and that it’s important to design the space towards this goal, with something of a collective responsibility for the feelings of safety of each individual.
I think Said things that individuals bear full responsibility their feelings of safety, and that it’s actively harmful to make these something the group space has to worry about. I think Said might even believe that “social safety” isn’t even important for the space, i.e., it’s fine if people actually are attacked in social ways, e.g. reputationally harm, caused to be punished by the group, made to experience negative feelings due to aggression from others.
----
If I had to choose between my model of mr-hire’s preferred space and my model of Said’s preferred space, I think I would actually choose Said’s. (Though I might not be correctly characterizing either—I wanted to state my prediction before I asked to test how successfully modeling other’s views).
When it comes to truth seeking, I’d rather err on the side of people getting harmed a bit and having to do a bunch of work to “steel” themselves against the “harsh” environment, then give individuals such a powerful tool (the space being responsible for their perception of being harmed) to disrupt and interfere with discourse. I know that’s not the intended result, but it seems too ripe for abuse to give feelings and needs the primacy I think is being given in the OP scenario. Something like an unachievable utopia: it sounds good, but I am very doubtful it can be done and also be a truth-seeking space.
[Also Said, I had a dream last night that I met you in Central Park, NY. I don’t know what you look or sound like in person, but I enjoyed meeting my dream version of you.]
Well, this is certainly not an egregious strawman by any stretch of the imagination—it’s a reasonable first approximation, really—but I would prefer to be somewhat more precise/nuanced. I would say this:
Individuals bear full responsibility for having their feelings (of safety, yes, and any other relevant propositional attitudes) match the reality as it in fact (objectively/intersubjectively verifiably) presents itself to them.[1]
This, essentially, transforms complaints of “feeling unsafe” into complaints of “being unsafe”; and that is something that we (whoever it is who constitute the “we” in any given case) can consider, and judge. If you’re actually made unsafe by some circumstance, well, maybe we want to do something about that, or prevent it. (Or maybe we don’t, of course. Likely it would depend on the details!) If you’re perfectly safe but you feel unsafe… that’s your own business; deal with it yourself![2]
The relevant questions, again, are about truth and justice. Is it acceptable for people to be reputationally harmed? Well, how is this happening? Certainly libel is not acceptable. Revealing private information about someone (e.g., about their sexual preferences) is not acceptable. Plenty of other things that might cause reputational harm aren’t acceptable. But if you reveal that I, let us say, falsified scientific data (and if this actually is the case), great reputational harm will be done to me; and this is entirely proper. The fact of the harm itself, in other words, is not dispositive.
Similarly for punishment—punishment is proper if it is just, improper otherwise.
As far as “negative feelings” go… “aggression” is a loaded word; what do you mean by it? Suppose that we are having an in-person debate, and you physically assault me; this is “aggression” that would, no doubt, make me “experience negative feelings”; it would also, obviously, be utterly unacceptable behavior. On the other hand, if you did nothing of the sort, but instead made some cutting remark, in which you subtly impugned my intelligence and good taste—is that “aggression”? Or what if you simply said “Said, you’re completely wrong about this, and mistaken in every particular”… aggression? Or not? I might “experience negative feelings” in each of these cases! But the question of whether any of these behaviors are acceptable, or not, does not hinge primarily on whether they could conceivably be described, in some sense, as “aggression”.
In short… when it comes to deciding what is good and what is bad—as with so many other things—precision is everything.
On this, we entirely agree. (And I would add that it is not simply ripe for abuse; it is, in fact, abused, and rampantly, in all cases I have seen.)
Central Park is certainly a pleasant place to meet anyone! I can only hope that, should we ever meet in fact, I live up to the standards set by my dream self…
“reality as it in fact (objectively/intersubjectively verifiably) presents itself to them”: By this somewhat convoluted turn of phrase I mean simply that it’s conceivable for someone to be deceived—made to perceive the facts erroneously, through adversarial action—in which case it would, obviously, be unreasonable to say that it’s entirely the victim’s responsibility to have their feelings about reality match actual reality instead of reality as they are able to discern it; nevertheless this is not license to say “well, this is what the reality feels like to me”, because “what should you reasonably conclude is the reality, given the facts that, as we can all see, are available to you” is something that may be determined and agreed upon, and in no sense is an individual incorrigible on that question.
Which, of course, does not mean that “what’s the best technique for dealing with feeling unsafe when you’re actually safe” isn’t a topic that the group might discuss.
Thanks for the precise and nuanced write-up, and for not objecting to my crude attempt to characterize your position.
Nothing in your views described here strikes me as gravely mistaken, it seems like a sensible norm set. I suspect that many of our disagreements appear once we attempt be precise around acceptable and not acceptable behaviors and how they are handled.
I agree that “aggression” is fuzzy and that simply causing negative emotions is certainly not the criteria by which to judge the acceptability of behavior. I used those terms to indicate/gesture rather than define.
I have a draft, Three ways to upset people with your speech, which attempts to differentiate between importantly different cases. I find myself looking forward to your comments on it once I finally publish it. I don’t think I would have said that a week ago, and I think it’s largely feeling safer with you, which is in turn the result of greater familiarity (I’ve never been active in the LW comments as much as in the last few weeks). I’m more calibrated about the significance of your words now, the degree of malice behind them (possibly not that much?), and even the defensible positions underlying them. I’ve also updated that it’s possible to have a pleasant and valuable exchange.
(I do not say these things because I wish to malign you with my prior beliefs about you, but because I think they’re useful and relevant information.)
Your warm response to my mentioning dream-meeting you made me feel warm (also learning your Myers Briggs type).
(Okay, now please forgive me for using all the above as part of an “argument”; I mean it all genuinely, but it seems to be a very concrete applied way to discuss topics that have been in the air of late.)
This gets us into some tricky questions I can place in your framework. I think it will take us (+ all the others) a fair bit of conversation to answer, but I’ll mention them here now to at least raise them. (Possibly just saying this because I’m away this week and plan not to be online much.)
My updates on you (if correct) suggest that largely Said’s comments do not threaten me much and I shouldn’t feel negative feelings as a result. Much of this is just how Said talks, and he’s still interested in honest debate, not just shutting you down with hostile talk. But my question is the “reality as it presents itself to me” you mentioned. The reality might be that Said is safe, but was I, given my priors and evidence available to me before, wrong to be afraid before I gained more information about how to interpret Said?
(Maybe I was, but this is not obvious.)
Is the acceptability of behavior determined by what the recipient reasonably could have believed (as judged by . . . ?) or by the actual reality. Or there are three possibilities even: 1) what I could have reasonably believed was the significance of your actions, 2) what you could have reasonably believed was the significance of your actions, 3) what the actual significance of your actions were (if this can even be defined sensibly).
It does seem somewhat unfair if the acceptability of your behavior is impacted by what I can reasonably believe. It also seems somewhat unfair that I should experience attack because I reasonably lacked information.
How do we handle all this? I don’t definitively know. Judging what is acceptable/reasonable/fair and how all different perspectives add up . . . it’s a mess that I don’t think gets better even with more attempt at precision. I mostly want to avoid having to judge.
This is in large part what intuitively pushes me towards wanting people to be proactive in avoiding misinterpretations and miscalibrations of other’s intent—so we don’t have to judge who was at fault. I want people to give people enough info that they correctly know even when I’m harsh, I still want them to feel safe. Mostly applies to people who don’t know me well. Once the evidence has accrued and you’re calibrated on what things mean, you require little “padding” (this is my version of Combat culture essentially), but you’ve got to accrue that evidence and establish the significance of actions with others first.
--
Phew, like everything else, that was longer than expected. I should really start expected everything to be long.
Curious if this provides any more clarity on my position (even if it’s not persuasive) and curious where you disagree with this treatment.
Here’s the couple of thousand words that fell out when I attempted to write up my thoughts re safety and community norms.
Link seems broken
Thanks, fixed! It’s a little bit repetitive with everything else I’ve written lately, but maybe I’m getting it clearer with each iteration.
I ended up having thoughts here that grew beyond the context (how to think about this feels related to how to think about depleted willpower). Wrote a shortform post.
My current best guess is that you’d get the best results (measured roughly “useful ideas generated and non-useful ideas pruned”) from a collection of norms where
a) people need to take responsibility for their own feelings of fear, and there is help/guidelines on how to go about that if you’re not very good at it yet, and
b) people also take responsibility for learning to the social and writing skills to avoid particularly obvious failure modes.
i.e. “I’m feeling defensive” shouldn’t be a get out of jail free card. (in particular, any request that someone change their communication behavior should come with a corresponding costly signal that you are working on improving your ability to listen while triggered)
And while I think I’ve believed this for a couple weeks, I don’t think I was doing the work to actually embody it, and I think that’s been a mistake I’ve been making.
I’m holding the frame you wrote on your shortform feed re defensiveness for a bit to see how I feel about it.
I’ve been trying to use the phrase ‘feeling of safety’ when it comes up but it has the unfortunate property that ‘aspiring rationalist’ had, where there isn’t a stable equilibrium where people reliably say the whole phrase.
I hereby proclaim that “feelings of safety” be shortened to “fafety.” The domain of worrying about fafety is now “fafety concerns.”
Problem solved. All in a day’s work.
Strong upvote
Updated to? This wording surprises me, because I’m having trouble forming a hypothesis as to what your earlier position could have been. (I’m afraid I haven’t studied your blogging corpus.) What else is this website for, exactly?
Instrumental rationality?
My steelman of this position is something like, “I favored focusing instrumental rationality because it seemed, well, useful. At the time I figured that this was just a different subject than epistemic rationality, & focusing on it would at worst mean less progress improving the accuracy of our beliefs. But in hindsight this involved allowing epistemics to get worse for the sake of more instrumental success. I’ve now updated towards that having been a bad tradeoff.”
How close is that?
Thanks! I’m not sure this is a place where steelmanning is quite the appropriate tool. My past self was optimized for being my past self, not being right. He was mostly just not trying to solve this question.
But, in this case, I think the best tool is more properly called “modeling people” and maybe “empathy”.
Things my past self cared about and/or believed included:
All the probability stuff feels too hard to think about, and it doesn’t seem like it’s really going to help me that much even if I put a lot of work into it. So for me personally, I’m just going to try to “remember base rates” and a few other simple heuristics and call it a day. I was glad other people took it more seriously though
Truth seems like one of many important things. What matters is getting things accomplished. (I’ve never been optimizing against truth, I have just prioritized other things. There’s been times where I, say, only put 20 minutes into checking an essay for being right, rather than 2 hours, when I had reason to suspect I might have had motivated reasoning.)
I thought (and still think, although less strongly and for more nuanced reasons) that the in person rationality community is unhealthy because it only selects for a few narrow types of person, who are min-maxed in a particular skillset. And I think the in person community is important (both for epistemic and instrumental reasons). It is important to be a community that doesn’t actively drive away people who bring other skills to the table.
I still roughly believe all that. The main update is that there should a) be dedicated spaces that focus on truthseeking as their [probably] sacred value, b) that LessWrong should be such a space. (But, as noted in Tensions in Truthseeking, there are still different tradeoffs you can make in your truthseeking frame, and I think it’s good to have spaces that have made different min-max tradeoffs to explore those tradeoffs. For example, there might be math-heavy spaces, there might be “blunt communication” spaces that optimize for directness, there might be feelings-heavy spaces that optimize for understanding and owning your internal state)
(I have made a bit of conceptual progress on probability stuff. I probably will never do real Bayesian Wizardry but I think grok it better now – I can follow some conversations I didn’t used to be able to follow and in some cases I can participate in and uphold norms that help others on their way to learning it better than I)
There is an interesting thing in all this space I recently re-read while perusing the old critiques of Gleb. A paraphrase of the linked comment is:
And, while I’m not sure, I think I might have held a frame somewhat like that (I don’t have clear memories of biting either particular bullet). But my current position is “effective altruists should hold to a high epistemic standard, even when marketing. But, learn to market well within those constraints.”
Okay, but I thought the idea was that instrumental rationality and epistemic rationality are very closely related. Two sides of the same coin, not two flavors of good thing that sometimes trade off against each other. That agents achieve their goals by means of building accurate models, and using those models to “search out paths through probability” that steer the world into the desired goal-state. If the models aren’t accurate, the instrumental probability-bending magic doesn’t work and cannot work.
Okay, but geez man, my past self had different beliefs. What do you want here? What is your incredulity here aiming to accomplish? If you can’t simulate the mind of a person who showed up on LessWrong with one set of beliefs and gradually updated their beliefs in a set of directions that are common on the site, I think you should prioritize learning to simulate other minds a bit
I genuinely feel incredulous and am trying to express what I’m actually thinking in clear language? I mean, it’s also totally going to be the case that the underlying generator of “genuinely felt incredulity” is no doubt going to be some sort of subconscious monkey-politics status move designed by evolution to make myself look good at the expense of others. It’s important to notice that! But the mere fact of having noticed that doesn’t make the feeling go away, and given that the feeling is there, it’s probably going to leak into my writing. I could expend more effort doing a complicated System-2 political calculation that tries to simulate you and strategically compute what words I should say in order to have the desired effect on you. But not only is that more work than saying what I’m actually thinking in clear language, I also expect it to result in worse writing. Use the native architecture!
I mean, if it’ll help, we can construct a narrative in which my emotion of incredulity that was designed by evolution to make me look good, actually makes me look bad in local social reality? That’s a win-win Pareto improvement: I don’t have to mutilate my natural writing style in the name of so-called “cooperative” norms, and you don’t have to let my monkey-politics brain get away with “winning” the interaction.
How about this? Incredulity is, definitionally, a failed prediction. The fact that I felt incredulous means that my monkey status instincts are systematically distorting my anticipations about the world, making me delusionally perceive things as “obvious” exactly when they’re things that I coincidentally happened to already know, and not because of their actual degree-of-obviousness as operationalized by what fraction of others know them. (And conversely, I’ll delusionally perceive things as “nonobvious” exactly when I coincidentally happened to not-know them.)
(Slaps forehead) Hello, Megan! Ten years into this “rationality” business, and here I am still making rookie mistakes like this! How dumb can I get?
Thanks, this is a good suggestion! I probably am below average at avoiding the typical mind fallacy. You should totally feel superior to me on this account!
I think there are separate worthwhile skills of “focus on learning empathy/modeling and let clear language flow from that”, and also “writing skills exist that are separate from epistemics” (such as brevity, which I think actually factors in here a bit)
Something that may not have been clear from my past discussion is that when I say “this could have been written in a way that was less triggering”, or something, I’m not (usually) meaning that to be a harsh criticism. Just, the sort of thing that you should say ‘ah, that makes sense. I will work on that’ for the future.
It’s actually not clear to me that I should work on that. As a professional hazard of my other career, I’m pretty used to people trying to use “You would be more persuasive if you were nicer” as an attempted silencing tactic; if I just believed everyone who told me that, I would never get anything done.