I definitely agree that there could exist perverse situations where there are instrumental tradeoffs to be made in truthseeking of the kind I and others have been suspicious of. For lack of a better term, let me call these “instrumentally epistemic” arguments: claims of the form, “X is true, but the consequences of saying it will actually result in less knowledge on net.” I can totally believe that some instrumentally epistemic arguments might hold. There’s nothing in my understanding of how the universe works that would prevent that kind of scenario from happening.
But in practice, with humans, I expect that a solid supermajority of real-world attempts to explicitly advocate for norm changes on “instrumentally epistemic” grounds are going to be utterly facilerationalizations with the (typically unconscious) motivation of justifying cowardice, intellectual dishonesty, ego-protection, &c.
I (somewhat apologetically) made an “instrumentally epistemic” argument in a private email thread recently, and Ben seemed super pissed in his reply (bold italics, incredulous tone, ”?!?!?!?!?!” punctuation). But the thing is—even if I might conceivably go on to defend a modified form of my original argument—I can’t blame Ben for using a pissed-off tone in his reply. “Instrumentally epistemic” arguments are an enormous red flag—an infrared flag thirty meters wide. Your prior should be that someone making an “instrumentally epistemic” argument can be usefully modeled as trying to undermine your perception of reality and metaphorically slash your tires (even if their conscious phonological loop never contains the explicit sentence “And now I’m going to try to undermine Ray Arnold’s perception of reality”).
Now, maybe that prior can be overcome for some arguments and some arguers! But the apparent failure of the one making the “instrumentally epistemic” argument to notice the thirty-meter red flag, is another red flag.
I don’t think the hospital example does the situation justice. The trade-off of choosing whether to spend money on a heart transplant or nurse salaries doesn’t seem analogous to choosing between truth and the occasional allegedly-instrumentally-epistemic lie (like reassuring your interlocutor that you respect them even when you don’t, in fact, respect them). Rather, it seems more closely analogous to choice of inquiry area (like whether to study truths about chemistry, or truths about biology), with “minutes of study time” as the resource to be allocated rather than dollars.
If we want a maximally charitable medical analogy for “instrumentally epistemic” lies, I would instead nominate chemotherapy, where we deliberately poison patients in the hope of hurting cancer cells more than healthy cells. Chemotherapy can be good if there’s solid evidence that you have a specific type of cancer that responds well to that specific type of chemotherapy. But you should probably check that people aren’t just trying to poison you!
I’m not advocating lying here, I’m advocating learning the communication skills necessary to a) actually get people to understand your point (which they’ll have a harder time with if they’re defensive), and b) not wasting dozens of hours unnecessarily (which could be better spent on figuring other things out).
[and to be clear, I also advocate gaining the courage to speak the truth even if your voice trembles, and be willing to fight for it when it’s important. Just, those aren’t the only skills a rationalist or a rationalist space needs. Listening, communicating clearly, avoiding triggering people’s “use language as politics mode”, and modeling minds and frames different from your own are key skills too]
Your prior should be that someone making an “instrumentally epistemic” argument can be usefully modeled as trying to undermine your perception of reality and metaphorically slash your tires (even if their conscious phonological loop never contains the explicit sentence “And now I’m going to try to undermine Ray Arnold’s perception of reality”).
Why do you think this prior is right?
But the apparent failure of the one making the “instrumentally epistemic” argument to notice the thirty-meter red flag, is another red flag.
This seems true only if your prior is so obviously right that one couldn’t disagree with it in good faith. I’m not convinced of this.
(As I mentioned I’m sympathetic to both sides of the debate here, but I find myself wanting to question your side more, because it seems to display a lot more certainty (along with associated signals such as exasperation and incredulity), which doesn’t seem justified to me.)
Thanks, I appreciate it a lot! You should be questioning my “side” as harshly as you see fit, because if you ask questions I can’t satisfactorily answer, then maybe my side is wrong, and I should be informed of this in order to become less wrong.
Why do you think this prior is right?
The mechanism by which saying true things leads to more knowledge is at least straightforward: you present arguments and evidence, and other people evaluate those arguments and evidence using the same general rules of reasoning that they use for everything else, and hopefully they learn stuff.
In order for saying true things to lead to less knowledge, we need to postulate some more complicated failure mode where some side-effect of speech disrupts the ordinary process of learning. I can totally believe that such failure modes exist, and even that they’re common. But lately I seem to be seeing a lot of arguments of the form, “Ah, but we need to coordinate in order to create norms that make everyone feel Safe, and only then can we seek truth.” And I just … really have trouble taking this seriously as a good faith argument rather than an attempt to collude to protect everyone’s feelings? Like, telling the truth is not a coordination problem? You can just unilaterally tell the truth.
associated signals such as exasperation and incredulity
But lately I seem to be seeing a lot of arguments of the form, “Ah, but we need to coordinate in order to create norms that make everyone feel Safe, and only then can we seek truth.” And I just … really have trouble taking this seriously as a good faith argument rather than an attempt to collude to protect everyone’s feelings?
I want to address something that I think is quite important in the context of this post, because I think you’re pattern matching the “let’s make a space where people’s needs are addressed,” to the standard social justice safe space, but there are actually 3 types of safe spaces, and the one you’re imagining is not related to the ones this post is talking about.
The social justice kind, where nobody is allowed to bring up arguments that make you feel unsafe, is the one you’re talking about. “We need to make everyone feel safe and can’t seek truth until we do that” is describing an environment where truth seeking is basically impossible. I think private spaces like that are important in a rationalist environment, because some people are fragile and need to heal before they can participate in truth seeking, but are almost never right for an organization that has the goal of seeking truth.
Then there’s the kind that this post is talking about. In this type of environment, it’s safe to say “This conversation is making me feel unsafe, so I need to leave”. It’s also safe to say “It feels like your need for safety is getting in the way of truthseeking” as well as for other people to push back on that if they think that this person’s need for safety is so great in this moment that we need to accommodate them for a bit and return to this topic later. I think the majority of public truth-seeking spaces would be served by adopting this type of safety, in lieu of something like Crocker’s rules.
Then there’s the third type of safe space. In this type of safe space, you can say “This topic is making me feel unsafe” and the expected response is “Awesome, then we’re going to keep throwing you in as many situations like this as possible, poke that emotional wound, and help you work through it so you can level up as an individual and we can level up as an organization.” In this case, the safety comes from the strict vetting procedures and strong culture that let you know that people poking your are sincere and skilled, and the people being poked have the emotional strength to deal with it. I think that a good majority of PRIVATE truth seeking spaces should strive to be this third type of safe space.
One of the mistakes I made in this post was conflate the second and third types of safe spaces, so for instance I posited a public space that also had radical transparency, which is really only a tool you should use in a culture with strong vetting. However, I definitely was not suggesting the first type of safe space, but I get the impression that that’s what you keep imagining.
In this type of environment, it’s safe to say “This conversation is making me feel unsafe, so I need to leave”.
I mean, in the case of a website that people use in their free time, you don’t necessarily even need an excuse: if you don’t find a conversation valuable (because it’s making you feel unsafe or for any other reason), you can just strong-downvote them and stop replying.
There was a recent case on Less Wrong where one of two reasons I gave for calling for end-of-conversation was that I was feeling “emotionally exhausted”, which seems similar to feeling unsafe. But that was me explaining why I didn’t feel like talking anymore. I definitely wasn’t saying that my interlocutor should give equal weight to his needs, my needs, and the needs of the forum of the whole. I don’t see how anyone is supposed to compute that.
I don’t see how anyone is supposed to compute that.
If your primary metaphor for thought is simple computations or mathematical functions, I can see how this would be very confusing, but I don’t think that’s actually the native architecture of our brains. Instead our brain is noticing patterns, creating reusable heuristics, and simulating other people using empathy.
When you look at the question using that native architecture, it becomes relatively simple to find a reasonable answer. This is the same way that we regularly find solutions to complex negotiations between multiple parties, or plan complex situations with multiple constraints, even though many of those tasks are naively uncomputable. The shared values and culture serve to make sure those heuristics are calibrated similarly between people.
Reply
When you look at the question using that native architecture, it becomes relatively simple to find a reasonable answer.
I don’t think “reasonable” is the correct word here. You keep assuming away the possibility of conflict. It’s easy to find a peaceful answer by simulating other people using empathy, if there’s nothing anyone cares about more than not rocking the boat. But what about the least convenient possible world where one party has Something to Protect which the other party doesn’t think is “reasonable”?
The shared values and culture serve to make sure those heuristics are calibrated similarly between people.
Riiiight, about that. The OP is about robust organizations in general without mentioning any specific organization, but given the three mentions of “truthseeking”, I’d like to talk about the special case of this website, and set it in the context of a previous discussion we’ve had.
I don’t think the OP is compatible with the shared values and culture established in Sequences-era Overcoming Bias and Less Wrong. I was there (first comment December 22, 2007). If the Less Wrong and “rationalist” brand names are now largely being held by a different culture with different values, I and the forces I represent have an interest in fighting to take them back.
Let me reply to your dialogue with another. To set the scene, I’ve been drafting a forthcoming post (working title: “Schelling Categories, and Simple Membership Tests”) in my nascent Sequence on the cognitive function of categories, which is to refer back to my post “The Univariate Fallacy”. Let’s suppose that by the time I finally get around to publishing “Schelling Categories” (like the Great Teacher, I suffer from writer’s molasses), the Jill from your dialogue has broken out of her simulation, instantiated herself in our universe, and joined the LW2 moderation team.
Jill: Zack, I’ve had another complaint—separate from the one in May—about your tendency to steer conversations towards divisive topics, and I’m going to ask you to tone it down a bit when on Frontpage posts.
Zack: What? Why? Wait, sorry—that was a rhetorical question, which I’ve been told is a violation of cooperative discourse norms. I think I can guess what motivated the complaint. But I want to hear you explain it.
Jill: Well, you mentioned this “univariate fallacy” again, and in the context of some things you’veTweeted, there was some concern that you were actually trying to allude to gender differences, which might make some community members of marginalized genders feel uncomfortable.
Zack: (aside) I’m guess I’m glad I didn’t keep calling it Lewontin’s fallacy.
(to Jill) So … you’re asking me to tone down the statistics blogging—on less wrong dot com—because some people who read what I write elsewhere can correctly infer that my motivation for thinking about this particular statistical phenomenon was because I needed it to help me make sense of an area of science I’ve been horrifiedlyfascinatedwith for the last fourteen years, and that scientific question might make some people feel uncomfortable?
Jill: Right. Truthseeking is very important. However, it’s clear that just choosing one value as sacred and not allowing for tradeoffs can lead to very dysfunctional belief systems. I believe you’ve pointed at a clear tension in our values as they’re currently stated: the tension between freedom of speech and truth, and the value of making a space that people actually want to have intellectual discussions at. I’m only asking you to give equal weight to your own needs, the needs of the people you’re interacting with, and the needs of the organization as a whole.
Zack: I said No. As a commenter on lesswrong.com, my duty and my only duty is to try to make—wait, scratch the “try”—to make contributions that advance the art of human rationality. I consider myself to have a moral responsibility to ignore the emotional needs of other commenters—and symmetrically, I think they have a moral responsibility to ignore mine.
Jill: I’d prefer that you be more charitable and work to steelman what I said.
Zack: If you think I’ve misunderstood what you’ve said, I’m happy to listen to you clarify whatever part you think I’m getting wrong. The point of the principle of charity is that people are motivated to strawman their interlocutors; reminding yourself to be “charitable” to others helps to correct for this bias. But to tell others to be charitable to you without giving them feedback about how, specifically, you think they’re misinterpreting what you said—that doesn’t make any sense; it’s like you’re just trying to mash an “Agree with me” button. I can’t say anything about what your conscious intent might be, but I don’t know how to model this behavior as being in good faith—and I feel the same way about this new complaint against me.
Zack: If by “contextualizing norms” you simply mean that what a speaker means needs to be partially understood from context, and is more than just what the sentence the speaker said means, then I agree—that’s just former Denver Broncos quarterback Brian Griese philosopher of language H. P. Grice’s theory of conversational implicature. But when I apply contextualizing norms to itself and look at the context around which “contextualizing norms” was coined, it sure looks like the entire point of the concept is to shut down ideologically inconvenient areas of inquiry. It’s certainly understandable. As far as the unwashed masses are concerned, it’s probably for the best. But it’s not what this website is about—and it’s not what I’m about. Not anymore. I am an aspiring epistemic rationalist. I don’t negotiate with emotional blackmailers, I don’t double-crux with Suicide Rock, and I’ve got Something to Protect.
Jill: (baffled) What could possibly incentivize you to be so unpragmatic?
I don’t think “reasonable” is the correct word here. You keep assuming away the possibility of conflict. It’s easy to find a peaceful answer by simulating other people using empathy, if there’s nothing anyone cares about more than not rocking the boat. But what about the least convenient possible world where one party has Something to Protect which the other party doesn’t think is “reasonable”?
Yes, if someone has values that are in fact incompatible with the culture of the organization, they shouldn’t be joining that organization. I thought that was clear in my previous statements, but it may in fact have not been. If every damn time their own values are at odds with what are best for the organization given its’ values, that’s an incompatible difference. They should either find a different organization, or try the archipeligo model. There are such thing as irreconcilable value differences.
I don’t think the OP is compatible with the shared values and culture established in Sequences-era Overcoming Bias and Less Wrong.
I agree. I think when that culture was established, the community was missing important concepts about motivated reasoning and truth seeking and chose values that were in fact not optimized for the ultimate goal of creating a community that could solve important problems.
I think it is in fact good to experiment with the norms you’re talking about from the original site, but I think many of those norms originally caused the site to decline and people to go elsewhere. Given my current mental models, I predict a site that uses those norms to make less intellectual progress than a similar site using my norms although I expect you to have the opposite intuition. As I stated in the introduction, the goal of this post was simply to make sure that those mental models were in discourse.
Re your dialogue: The main thing that I got from it was that you think a lot of the arguments in the OP are motivated reasoning and will lead to bad incentives. I also got that this is a subject you care a lot about.
I think when that culture was established, the community was missing important concepts about motivated reasoning and truth seeking
Can you be more specific? Can you name three specific concepts about motivated reasoning and truthseeking that you know, but Sequences-era Overcoming Bias/Less Wrong didn’t?
I think many of those norms originally caused the site to decline and people to go elsewhere.
I mean, that’s one hypothesis. In contrast, my model has been that communities congregate around predictable sources of high-quality writing, and people who can produce high-quality content in high volume are very rare. Thus, once Eliezer Yudkowsky stopped being active, and Yvain a.k.a. the immortal Scott Alexander moved to Slate Star Codex (in part so that he could write about politics, which we’ve traditionally avoided), all the “intellectual energy” followed Scott to SSC.
Can you think of any testable predictions (or retrodictions) that would distinguish my model from your model?
I also got that this is a subject you care a lot about.
Can you be more specific? Can you name three specific concepts about motivated reasoning and truthseeking that you know, but Sequences-era Overcoming Bias/Less Wrong didn’t?
Here are a few:
The importance of creating a culture that develops Kegan 5 leaders that can take over for the current leaders and help meaningfully change the values as the context changes, in a way that doesn’t simply cause organizations to value drift along with the current broader culture.
How ignoring or not attending for people’s needs creates incentives for motivated reasoning, and how to create spaces that get rid of those incentives WITHOUT being hijacked by whoever screams the loudest.
The importance of cultural tradition and ritual in embedding concepts in augmenting the teaching and telling people what concepts are important.
Can you think of any testable predictions (or retrodictions) that would distinguish my model from your model?
No because I think that our models are compatible. My model is about how to attract, retain, and develop people with high potential or skill that are in alignment your community’s values, and your model says that not retaining, attracting, or developing people that matched our communities values and had high writing skill is what caused it to fail.
If you can give a specific model of why LW1 failed to attract, retain, and develop high quality writers, then I think there’s a better space for comparison. Perhaps you can also point out some testable predictions that each of our models would make.
In contrast, my model has been that communities congregate around predictable sources of high-quality writing, and people who can produce high-quality content in high volume are very rare. Thus, once Eliezer Yudkowsky stopped being active, and Yvain a.k.a. the immortal Scott Alexander moved to Slate Star Codex (in part so that he could write about politics, which we’ve traditionally avoided), all the “intellectual energy” followed Scott to SSC.
First, I want to state that I agree with this model. However, I also want to note that the SSC comments section tend to have fairly low-quality discussion (in comparison to the OB/LW 1.0 heyday), and I’m not sure why this is; candidate hypotheses include that Scott’s explicit politics attracted people with lower epistemic standards, or that the lack of an explicit karma system allowed low-quality discussion to persist (but I don’t think OB had an explicit karma system either?).
Overall, I’m unsure as to what kind of norms/technology maintains high-quality discussion (as opposed to just the presence of discussion in general), and it’s plausible to me that the two may actually be somewhat mutually exclusive (in the sense that norms/technology designed to promote the volume of high-quality discussion may in fact reduce the volume of discussion in general). It’s not clear to me how this tradeoff should be balanced.
in part so that he could write about politics, which we’ve traditionally avoided
I want to state that I agree with this model.
(I sometimes think that I might be well-positioned to fill the market niche that Scott occupied in 2014, but no longer can due to his being extortable (“As I became more careful in my own writings [...]”) in a way that I have been trained not to be. But I would need to learn to write faster.)
One thing is that I think early OBNYC and LW just actually had a lot of chaff comments too. I think people disproportionately remember the great parts.
When you look at the question using that native architecture, it becomes relatively simple to find a reasonable answer. This is the same way that we regularly find solutions to complex negotiations between multiple parties, or plan complex situations with multiple constraints, even though many of those tasks are naively uncomputable.
I’m not confident that it does. I perhaps expect people doing this using the native architecture to feel like they’ve found a reasonable answer. But I would expect them to actually be prioritising their own feelings, in most cases. (Though some people will underweight their own feelings. And perhaps some people will get it right.)
Perhaps they will get close enough for the answer to still count as “reasonable”?
If someone attempts to give equal weight to their own needs, the meds of their interlocutor, and the needs of the forum as a whole—how do we know whether they’ve got a reasonable answer? Does that just have to be left to moderator discretion, or?
If someone attempts to give equal weight to their own needs, the meds of their interlocutor, and the needs of the forum as a whole—how do we know whether they’ve got a reasonable answer? Does that just have to be left to moderator discretion, or?
Yes basically, but if the forum were to take on this direction, the idea would be to have enough case examples/explanations from the moderators about WHY they made that discretion to calibrate people’s reasonable answers. See also this response to Zach which goes more into details about the systems in place to calibrate people’s reasonable answers.
I’m rather confused about what you mean by ‘safe’. I thought I knew what the word meant, but the way you (and some others) are using it perplexes me. Could you explain how to interpret this notion of “safety”?
For instance, this part:
Then there’s the kind that this post is talking about. In this type of environment, it’s safe to say “This conversation is making me feel unsafe, so I need to leave”. It’s also safe to say “It feels like your need for safety is getting in the way of truthseeking” as well as for other people to push back on that if they think that this person’s need for safety is so great in this moment that we need to accommodate them for a bit and return to this topic later.
[emphasis mine]
What do the bolded uses of ‘safe’ mean?
Is it the same meaning as the other uses of ‘safe’ in your comment? If not, what other meanings are in use, in which parts of the comment?
I think it’s best defined by its’ antonym. Unsafety, in this context, would mean anything that triggers a defensive or reactive reaction. Just like how bodily unsafety triggers fear, agression, etc, there are psychological equivalents that trigger the same reaction.
Safety is when a particualr circumstance doesn’t trigger that reaction, OR alternatively there could be a meta safety (AKA, having that reaction doesn’t trigger that reaction, because it’s ok).
I think your bolded definitions of safe would actually be served by changing to the word allowed, which for many people correlates quite closely with their feeling of safety.
I think the question of “what is safety?” is a really good one. I’ll write up some thoughts here both for this thread, but also to be to refer to generally (hence a bit more length).
Safety is when a particular circumstance doesn’t trigger that reaction,
I’m not a fan of that definition. It’s equating “feelings of safety” with “actual safety”
It’s defining safety as the absence the response to perceived unsafety. It feels equivalent to saying “sickness is the thing your immune system fights, and health is the absence of your immune system being triggered to fight something.” Which is very approximately true, but breaks down when you consider autoimmune disorders. With those, it’s the mistaken perception of attack which is the very problem.
This definition can also put a lot of the power in the hands of those who are having a reaction. If we all agree that our conversation must be safe, and that any individual can declare it unsafe because they are having a reaction, this gives a lot power to individuals to force attention on the question of safety (and I fear too asymmetrically with others being blamed for causing the feelings of uncertainty).
-----
So here’s the alternative positive account of “safety” I would give:
One *is* safe if one is unlikely to be harmed; one *feels* if they believe (S1 and/or S2) if they believe they won’t be harmed.
This accords with the standard use of safety, e.g. safety goggles, safety precautions, safe neighborhood, etc.
In conversation, one can be “harmed socially”, e.g. excluded from the group, being “punished” by the group, being made to look bad or stupid (with consequences on how they are treated), having someone act hostilely or aggressive to them (which is a risk of strong negative experience even if they S2 believe it won’t come to any physical or lasting harm), etc. (this is not a carefully developed or complete list).
So in conversation and social spaces, safety equates to not being likely to be harmed in the above ways.
Much the same defenses that activate when feeling under physical threat also come online when feeling under social threat (for indeed, both can be very risky to a human). These are physiological states, fight or flight, etc. How adaptative these are in the modern age . . . more than 0, less than 1 . . .? Having these responses indicates that some part of your mind perceives threat, the question being whether it’s calibrated.
On the question of space: a space can be perceived to have or lower risk of harm to individual (safety) and also higher or lower assessments of risk of harm related to taking specific actions, e.g. saying certain things.
---
With this definition, we can separately evaluate the questions of:
1) Are people actually safe vs likely to be harmed in various ways?
2) Are the harms people worried actually legitimate harms to be worried about?
3) Are people correct to be afraid of being harmed, that is, to feel unsafe?
4) Who should be taking action to cause people to feel unsafe? How is responsibility distributed between the individual and the group?
5) How much should the group/community worry about a) actual safety, and b) perceived safety?
I’m interested in how different people answer these questions generally and in the context of LessWrong.
> I’m not a fan of that definition. It’s equating “feelings of safety” with “actual safety”
I agree with this, but it’s quite a mouthful to deal with. And I think “feelings of safety” are actually more important for truthseeking and creating a product—they’re the things that produce defensiveness, motivated reasoning, etc.
I think mr-hire thinks the important success condition is that people feel safe and that it’s important to design the space towards this goal, with something of a collective responsibility for the feelings of safety of each individual.
This seems rightish- but off in really important ways that I can’t articulate. It’s putting the emphasis on the wrong things and “collective responsiblity” is not an idea I like at all. I think I’d put my stance as something like “feeling unsafe is a major driver of what people say and do, and good cultures provide space to process and deal with those feelings of unsafety”
This definition can also put a lot of the power in the hands of those who are having a reaction. If we all agree that our conversation must be safe, and that any individual can declare it unsafe because they are having a reaction, this gives a lot power to individuals to force attention on the question of safety (and I fear too asymmetrically with others being blamed for causing the feelings of uncertainty).
Note that this issue is explicitly addressed in the original dialogue. If someones feelings are hurting the discourse, they need to take responsibility for that just as much as I need to take responsibility for hurting their feelings. No one is agreeing that all conversations must be safe for all people, but simply that taking into account when people feel unsafe is important.
I agree with this, but it’s quite a mouthful to deal with
Yeah, but there’s a really big difference! You can’t give up that precision.
This seems rightish- but off in really important ways that I can’t articulate.
Nods. Also agree that “collective responsibility” is not the most helpful concept to talk about.
Note that this issue is explicitly addressed in the original dialogue. If someones feelings are hurting the discourse, they need to take responsibility for that just as much as I need to take responsibility for hurting their feelings.
Indeed, the fact people can say “”It feels like your need for safety is getting in the way of truth-seeking”is crucial for it to have any chance.
My expectation based on related real-life experience though, is that if making your need for safety is an option, there will people who abuse this and use it to suck up a lot of time and attention. That technically someone could deny their claim and move on, but this will happen much later than optimal and in the meantime everyone’s attention has been sucked into a great drama. Attempts to say “your safety is disrupting truth-seeking” are accused as being attempts to oppress someone, etc.
This is all imagining how it would go with typical humans. I’m guessing you’re imagining better-than-typical people in your org who won’t have the same failure mode, so maybe it’ll be fine. I’m mostly anchored how I expect that approach to go if applied to most humans I’ve known (especially those really into caring about feelings and who’d be likely to sign up for it).
I think mr-hire thinks the important success condition is that people feel safe and that it’s important to design the space towards this goal, with something of a collective responsibility for the feelings of safety of each individual.
I think Said things that individuals bear full responsibility their feelings of safety, and that it’s actively harmful to make these something the group space has to worry about. I think Said might even believe that “social safety” isn’t even important for the space, i.e., it’s fine if people actually are attacked in social ways, e.g. reputationally harm, caused to be punished by the group, made to experience negative feelings due to aggression from others.
----
If I had to choose between my model of mr-hire’s preferred space and my model of Said’s preferred space, I think I would actually choose Said’s. (Though I might not be correctly characterizing either—I wanted to state my prediction before I asked to test how successfully modeling other’s views).
When it comes to truth seeking, I’d rather err on the side of people getting harmed a bit and having to do a bunch of work to “steel” themselves against the “harsh” environment, then give individuals such a powerful tool (the space being responsible for their perception of being harmed) to disrupt and interfere with discourse. I know that’s not the intended result, but it seems too ripe for abuse to give feelings and needs the primacy I think is being given in the OP scenario. Something like an unachievable utopia: it sounds good, but I am very doubtful it can be done and also be a truth-seeking space.
[Also Said, I had a dream last night that I met you in Central Park, NY. I don’t know what you look or sound like in person, but I enjoyed meeting my dream version of you.]
I think Said things that individuals bear full responsibility their feelings of safety, and that it’s actively harmful to make these something the group space has to worry about.
Well, this is certainly not an egregious strawman by any stretch of the imagination—it’s a reasonable first approximation, really—but I would prefer to be somewhat more precise/nuanced. I would say this:
Individuals bear full responsibility for having their feelings (of safety, yes, and any other relevant propositional attitudes) match the reality as it in fact (objectively/intersubjectively verifiably) presents itself to them.[1]
This, essentially, transforms complaints of “feeling unsafe” into complaints of “being unsafe”; and that is something that we (whoever it is who constitute the “we” in any given case) can consider, and judge. If you’re actually made unsafe by some circumstance, well, maybe we want to do something about that, or prevent it. (Or maybe we don’t, of course. Likely it would depend on the details!) If you’re perfectly safe but you feel unsafe… that’s your own business; deal with it yourself![2]
I think Said might even believe that “social safety” isn’t even important for the space, i.e., it’s fine if people actually are attacked in social ways, e.g. reputationally harm, caused to be punished by the group, made to experience negative feelings due to aggression from others.
The relevant questions, again, are about truth and justice. Is it acceptable for people to be reputationally harmed? Well, how is this happening? Certainly libel is not acceptable. Revealing private information about someone (e.g., about their sexual preferences) is not acceptable. Plenty of other things that might cause reputational harm aren’t acceptable. But if you reveal that I, let us say, falsified scientific data (and if this actually is the case), great reputational harm will be done to me; and this is entirely proper. The fact of the harm itself, in other words, is not dispositive.
Similarly for punishment—punishment is proper if it is just, improper otherwise.
As far as “negative feelings” go… “aggression” is a loaded word; what do you mean by it? Suppose that we are having an in-person debate, and you physically assault me; this is “aggression” that would, no doubt, make me “experience negative feelings”; it would also, obviously, be utterly unacceptable behavior. On the other hand, if you did nothing of the sort, but instead made some cutting remark, in which you subtly impugned my intelligence and good taste—is that “aggression”? Or what if you simply said “Said, you’re completely wrong about this, and mistaken in every particular”… aggression? Or not? I might “experience negative feelings” in each of these cases! But the question of whether any of these behaviors are acceptable, or not, does not hinge primarily on whether they could conceivably be described, in some sense, as “aggression”.
In short… when it comes to deciding what is good and what is bad—as with so many other things—precision is everything.
When it comes to truth seeking, I’d rather err on the side of people getting harmed a bit and having to do a bunch of work to “steel” themselves against the “harsh” environment, then give individuals such a powerful tool (the space being responsible for their perception of being harmed) to disrupt and interfere with discourse. I know that’s not the intended result, but it seems too ripe for abuse to give feelings and needs the primacy I think is being given in the OP scenario.
On this, we entirely agree. (And I would add that it is not simply ripe for abuse; it is, in fact, abused, and rampantly, in all cases I have seen.)
[Also Said, I had a dream last night that I met you in Central Park, NY. I don’t know what you look or sound like in person, but I enjoyed meeting my dream version of you.]
Central Park is certainly a pleasant place to meet anyone! I can only hope that, should we ever meet in fact, I live up to the standards set by my dream self…
“reality as it in fact (objectively/intersubjectively verifiably) presents itself to them”: By this somewhat convoluted turn of phrase I mean simply that it’s conceivable for someone to be deceived—made to perceive the facts erroneously, through adversarial action—in which case it would, obviously, be unreasonable to say that it’s entirely the victim’s responsibility to have their feelings about reality match actual reality instead of reality as they are able to discern it; nevertheless this is not license to say “well, this is what the reality feels like to me”, because “what should you reasonably conclude is the reality, given the facts that, as we can all see, are available to you” is something that may be determined and agreed upon, and in no sense is an individual incorrigible on that question.
Which, of course, does not mean that “what’s the best technique for dealing with feeling unsafe when you’re actually safe” isn’t a topic that the group might discuss.
Thanks for the precise and nuanced write-up, and for not objecting to my crude attempt to characterize your position.
Nothing in your views described here strikes me as gravely mistaken, it seems like a sensible norm set. I suspect that many of our disagreements appear once we attempt be precise around acceptable and not acceptable behaviors and how they are handled.
I agree that “aggression” is fuzzy and that simply causing negative emotions is certainly not the criteria by which to judge the acceptability of behavior. I used those terms to indicate/gesture rather than define.
I have a draft, Three ways to upset people with your speech, which attempts to differentiate between importantly different cases. I find myself looking forward to your comments on it once I finally publish it. I don’t think I would have said that a week ago, and I think it’s largely feeling safer with you, which is in turn the result of greater familiarity (I’ve never been active in the LW comments as much as in the last few weeks). I’m more calibrated about the significance of your words now, the degree of malice behind them (possibly not that much?), and even the defensible positions underlying them. I’ve also updated that it’s possible to have a pleasant and valuable exchange.
(I do not say these things because I wish to malign you with my prior beliefs about you, but because I think they’re useful and relevant information.)
Your warm response to my mentioning dream-meeting you made me feel warm (also learning your Myers Briggs type).
(Okay, now please forgive me for using all the above as part of an “argument”; I mean it all genuinely, but it seems to be a very concrete applied way to discuss topics that have been in the air of late.)
This gets us into some tricky questions I can place in your framework. I think it will take us (+ all the others) a fair bit of conversation to answer, but I’ll mention them here now to at least raise them. (Possibly just saying this because I’m away this week and plan not to be online much.)
My updates on you (if correct) suggest that largely Said’s comments do not threaten me much and I shouldn’t feel negative feelings as a result. Much of this is just how Said talks, and he’s still interested in honest debate, not just shutting you down with hostile talk. But my question is the “reality as it presents itself to me” you mentioned. The reality might be that Said is safe, but was I, given my priors and evidence available to me before, wrong to be afraid before I gained more information about how to interpret Said?
(Maybe I was, but this is not obvious.)
Is the acceptability of behavior determined by what the recipient reasonably could have believed (as judged by . . . ?) or by the actual reality. Or there are three possibilities even: 1) what I could have reasonably believed was the significance of your actions, 2) what you could have reasonably believed was the significance of your actions, 3) what the actual significance of your actions were (if this can even be defined sensibly).
It does seem somewhat unfair if the acceptability of your behavior is impacted by what I can reasonably believe. It also seems somewhat unfair that I should experience attack because I reasonably lacked information.
How do we handle all this? I don’t definitively know. Judging what is acceptable/reasonable/fair and how all different perspectives add up . . . it’s a mess that I don’t think gets better even with more attempt at precision. I mostly want to avoid having to judge.
This is in large part what intuitively pushes me towards wanting people to be proactive in avoiding misinterpretations and miscalibrations of other’s intent—so we don’t have to judge who was at fault. I want people to give people enough info that they correctly know even when I’m harsh, I still want them to feel safe. Mostly applies to people who don’t know me well. Once the evidence has accrued and you’re calibrated on what things mean, you require little “padding” (this is my version of Combat culture essentially), but you’ve got to accrue that evidence and establish the significance of actions with others first.
--
Phew, like everything else, that was longer than expected. I should really start expected everything to be long.
Curious if this provides any more clarity on my position (even if it’s not persuasive) and curious where you disagree with this treatment.
I ended up having thoughts here that grew beyond the context (how to think about this feels related to how to think about depleted willpower). Wrote a shortform post.
My current best guess is that you’d get the best results (measured roughly “useful ideas generated and non-useful ideas pruned”) from a collection of norms where
a) people need to take responsibility for their own feelings of fear, and there is help/guidelines on how to go about that if you’re not very good at it yet, and
b) people also take responsibility for learning to the social and writing skills to avoid particularly obvious failure modes.
i.e. “I’m feeling defensive” shouldn’t be a get out of jail free card. (in particular, any request that someone change their communication behavior should come with a corresponding costly signal that you are working on improving your ability to listen while triggered)
And while I think I’ve believed this for a couple weeks, I don’t think I was doing the work to actually embody it, and I think that’s been a mistake I’ve been making.
I’ve been trying to use the phrase ‘feeling of safety’ when it comes up but it has the unfortunate property that ‘aspiring rationalist’ had, where there isn’t a stable equilibrium where people reliably say the whole phrase.
I definitely agree that there could exist perverse situations where there are instrumental tradeoffs to be made in truthseeking of the kind I and others have been suspicious of. For lack of a better term, let me call these “instrumentally epistemic” arguments: claims of the form, “X is true, but the consequences of saying it will actually result in less knowledge on net.” I can totally believe that some instrumentally epistemic arguments might hold. There’s nothing in my understanding of how the universe works that would prevent that kind of scenario from happening.
But in practice, with humans, I expect that a solid supermajority of real-world attempts to explicitly advocate for norm changes on “instrumentally epistemic” grounds are going to be utterly facile rationalizations with the (typically unconscious) motivation of justifying cowardice, intellectual dishonesty, ego-protection, &c.
I (somewhat apologetically) made an “instrumentally epistemic” argument in a private email thread recently, and Ben seemed super pissed in his reply (bold italics, incredulous tone, ”?!?!?!?!?!” punctuation). But the thing is—even if I might conceivably go on to defend a modified form of my original argument—I can’t blame Ben for using a pissed-off tone in his reply. “Instrumentally epistemic” arguments are an enormous red flag—an infrared flag thirty meters wide. Your prior should be that someone making an “instrumentally epistemic” argument can be usefully modeled as trying to undermine your perception of reality and metaphorically slash your tires (even if their conscious phonological loop never contains the explicit sentence “And now I’m going to try to undermine Ray Arnold’s perception of reality”).
Now, maybe that prior can be overcome for some arguments and some arguers! But the apparent failure of the one making the “instrumentally epistemic” argument to notice the thirty-meter red flag, is another red flag.
I don’t think the hospital example does the situation justice. The trade-off of choosing whether to spend money on a heart transplant or nurse salaries doesn’t seem analogous to choosing between truth and the occasional allegedly-instrumentally-epistemic lie (like reassuring your interlocutor that you respect them even when you don’t, in fact, respect them). Rather, it seems more closely analogous to choice of inquiry area (like whether to study truths about chemistry, or truths about biology), with “minutes of study time” as the resource to be allocated rather than dollars.
If we want a maximally charitable medical analogy for “instrumentally epistemic” lies, I would instead nominate chemotherapy, where we deliberately poison patients in the hope of hurting cancer cells more than healthy cells. Chemotherapy can be good if there’s solid evidence that you have a specific type of cancer that responds well to that specific type of chemotherapy. But you should probably check that people aren’t just trying to poison you!
I’m not advocating lying here, I’m advocating learning the communication skills necessary to a) actually get people to understand your point (which they’ll have a harder time with if they’re defensive), and b) not wasting dozens of hours unnecessarily (which could be better spent on figuring other things out).
[and to be clear, I also advocate gaining the courage to speak the truth even if your voice trembles, and be willing to fight for it when it’s important. Just, those aren’t the only skills a rationalist or a rationalist space needs. Listening, communicating clearly, avoiding triggering people’s “use language as politics mode”, and modeling minds and frames different from your own are key skills too]
Why do you think this prior is right?
This seems true only if your prior is so obviously right that one couldn’t disagree with it in good faith. I’m not convinced of this.
(As I mentioned I’m sympathetic to both sides of the debate here, but I find myself wanting to question your side more, because it seems to display a lot more certainty (along with associated signals such as exasperation and incredulity), which doesn’t seem justified to me.)
Thanks, I appreciate it a lot! You should be questioning my “side” as harshly as you see fit, because if you ask questions I can’t satisfactorily answer, then maybe my side is wrong, and I should be informed of this in order to become less wrong.
The mechanism by which saying true things leads to more knowledge is at least straightforward: you present arguments and evidence, and other people evaluate those arguments and evidence using the same general rules of reasoning that they use for everything else, and hopefully they learn stuff.
In order for saying true things to lead to less knowledge, we need to postulate some more complicated failure mode where some side-effect of speech disrupts the ordinary process of learning. I can totally believe that such failure modes exist, and even that they’re common. But lately I seem to be seeing a lot of arguments of the form, “Ah, but we need to coordinate in order to create norms that make everyone feel Safe, and only then can we seek truth.” And I just … really have trouble taking this seriously as a good faith argument rather than an attempt to collude to protect everyone’s feelings? Like, telling the truth is not a coordination problem? You can just unilaterally tell the truth.
Hm, I think there’s a risk of signal miscalibration here. Just because I feel exasperated and this emotion leaks into my writing, doesn’t necessarily mean implied probabilities close to 1? (Related: Say It Loud. See also my speculative just-so story about why the incredulity is probably non-normative.)
(It’s 1:20 a.m. on Sunday and I’ve used up my internet quota for the weekend, so it might take me a few days to respond to future comments.)
I want to address something that I think is quite important in the context of this post, because I think you’re pattern matching the “let’s make a space where people’s needs are addressed,” to the standard social justice safe space, but there are actually 3 types of safe spaces, and the one you’re imagining is not related to the ones this post is talking about.
The social justice kind, where nobody is allowed to bring up arguments that make you feel unsafe, is the one you’re talking about. “We need to make everyone feel safe and can’t seek truth until we do that” is describing an environment where truth seeking is basically impossible. I think private spaces like that are important in a rationalist environment, because some people are fragile and need to heal before they can participate in truth seeking, but are almost never right for an organization that has the goal of seeking truth.
Then there’s the kind that this post is talking about. In this type of environment, it’s safe to say “This conversation is making me feel unsafe, so I need to leave”. It’s also safe to say “It feels like your need for safety is getting in the way of truthseeking” as well as for other people to push back on that if they think that this person’s need for safety is so great in this moment that we need to accommodate them for a bit and return to this topic later. I think the majority of public truth-seeking spaces would be served by adopting this type of safety, in lieu of something like Crocker’s rules.
Then there’s the third type of safe space. In this type of safe space, you can say “This topic is making me feel unsafe” and the expected response is “Awesome, then we’re going to keep throwing you in as many situations like this as possible, poke that emotional wound, and help you work through it so you can level up as an individual and we can level up as an organization.” In this case, the safety comes from the strict vetting procedures and strong culture that let you know that people poking your are sincere and skilled, and the people being poked have the emotional strength to deal with it. I think that a good majority of PRIVATE truth seeking spaces should strive to be this third type of safe space.
One of the mistakes I made in this post was conflate the second and third types of safe spaces, so for instance I posited a public space that also had radical transparency, which is really only a tool you should use in a culture with strong vetting. However, I definitely was not suggesting the first type of safe space, but I get the impression that that’s what you keep imagining.
I mean, in the case of a website that people use in their free time, you don’t necessarily even need an excuse: if you don’t find a conversation valuable (because it’s making you feel unsafe or for any other reason), you can just strong-downvote them and stop replying.
There was a recent case on Less Wrong where one of two reasons I gave for calling for end-of-conversation was that I was feeling “emotionally exhausted”, which seems similar to feeling unsafe. But that was me explaining why I didn’t feel like talking anymore. I definitely wasn’t saying that my interlocutor should give equal weight to his needs, my needs, and the needs of the forum of the whole. I don’t see how anyone is supposed to compute that.
If your primary metaphor for thought is simple computations or mathematical functions, I can see how this would be very confusing, but I don’t think that’s actually the native architecture of our brains. Instead our brain is noticing patterns, creating reusable heuristics, and simulating other people using empathy.
When you look at the question using that native architecture, it becomes relatively simple to find a reasonable answer. This is the same way that we regularly find solutions to complex negotiations between multiple parties, or plan complex situations with multiple constraints, even though many of those tasks are naively uncomputable. The shared values and culture serve to make sure those heuristics are calibrated similarly between people. Reply
I don’t think “reasonable” is the correct word here. You keep assuming away the possibility of conflict. It’s easy to find a peaceful answer by simulating other people using empathy, if there’s nothing anyone cares about more than not rocking the boat. But what about the least convenient possible world where one party has Something to Protect which the other party doesn’t think is “reasonable”?
Riiiight, about that. The OP is about robust organizations in general without mentioning any specific organization, but given the three mentions of “truthseeking”, I’d like to talk about the special case of this website, and set it in the context of a previous discussion we’ve had.
I don’t think the OP is compatible with the shared values and culture established in Sequences-era Overcoming Bias and Less Wrong. I was there (first comment December 22, 2007). If the Less Wrong and “rationalist” brand names are now largely being held by a different culture with different values, I and the forces I represent have an interest in fighting to take them back.
Let me reply to your dialogue with another. To set the scene, I’ve been drafting a forthcoming post (working title: “Schelling Categories, and Simple Membership Tests”) in my nascent Sequence on the cognitive function of categories, which is to refer back to my post “The Univariate Fallacy”. Let’s suppose that by the time I finally get around to publishing “Schelling Categories” (like the Great Teacher, I suffer from writer’s molasses), the Jill from your dialogue has broken out of her simulation, instantiated herself in our universe, and joined the LW2 moderation team.
Jill: Zack, I’ve had another complaint—separate from the one in May—about your tendency to steer conversations towards divisive topics, and I’m going to ask you to tone it down a bit when on Frontpage posts.
Zack: What? Why? Wait, sorry—that was a rhetorical question, which I’ve been told is a violation of cooperative discourse norms. I think I can guess what motivated the complaint. But I want to hear you explain it.
Jill: Well, you mentioned this “univariate fallacy” again, and in the context of some things you’ve Tweeted, there was some concern that you were actually trying to allude to gender differences, which might make some community members of marginalized genders feel uncomfortable.
Zack: (aside) I’m guess I’m glad I didn’t keep calling it Lewontin’s fallacy.
(to Jill) So … you’re asking me to tone down the statistics blogging—on less wrong dot com—because some people who read what I write elsewhere can correctly infer that my motivation for thinking about this particular statistical phenomenon was because I needed it to help me make sense of an area of science I’ve been horrifiedly fascinated with for the last fourteen years, and that scientific question might make some people feel uncomfortable?
Jill: Right. Truthseeking is very important. However, it’s clear that just choosing one value as sacred and not allowing for tradeoffs can lead to very dysfunctional belief systems. I believe you’ve pointed at a clear tension in our values as they’re currently stated: the tension between freedom of speech and truth, and the value of making a space that people actually want to have intellectual discussions at. I’m only asking you to give equal weight to your own needs, the needs of the people you’re interacting with, and the needs of the organization as a whole.
Zack: (aside) Wow. It’s like I’m actually living in Atlas Shrugged, just like Michael Vassar said. (to Jill) No.
Jill: What?
Zack: I said No. As a commenter on lesswrong.com, my duty and my only duty is to try to make—wait, scratch the “try”—to make contributions that advance the art of human rationality. I consider myself to have a moral responsibility to ignore the emotional needs of other commenters—and symmetrically, I think they have a moral responsibility to ignore mine.
Jill: I’d prefer that you be more charitable and work to steelman what I said.
Zack: If you think I’ve misunderstood what you’ve said, I’m happy to listen to you clarify whatever part you think I’m getting wrong. The point of the principle of charity is that people are motivated to strawman their interlocutors; reminding yourself to be “charitable” to others helps to correct for this bias. But to tell others to be charitable to you without giving them feedback about how, specifically, you think they’re misinterpreting what you said—that doesn’t make any sense; it’s like you’re just trying to mash an “Agree with me” button. I can’t say anything about what your conscious intent might be, but I don’t know how to model this behavior as being in good faith—and I feel the same way about this new complaint against me.
Jill: Contextualizing norms are valid rationality norms!
Zack: If by “contextualizing norms” you simply mean that what a speaker means needs to be partially understood from context, and is more than just what the sentence the speaker said means, then I agree—that’s just
former Denver Broncos quarterback Brian Griesephilosopher of language H. P. Grice’s theory of conversational implicature. But when I apply contextualizing norms to itself and look at the context around which “contextualizing norms” was coined, it sure looks like the entire point of the concept is to shut down ideologically inconvenient areas of inquiry. It’s certainly understandable. As far as the unwashed masses are concerned, it’s probably for the best. But it’s not what this website is about—and it’s not what I’m about. Not anymore. I am an aspiring epistemic rationalist. I don’t negotiate with emotional blackmailers, I don’t double-crux with Suicide Rock, and I’ve got Something to Protect.Jill: (baffled) What could possibly incentivize you to be so unpragmatic?
Zack: It’s not the incentives! (aside) It’s me!
(Curtain.)
Yes, if someone has values that are in fact incompatible with the culture of the organization, they shouldn’t be joining that organization. I thought that was clear in my previous statements, but it may in fact have not been. If every damn time their own values are at odds with what are best for the organization given its’ values, that’s an incompatible difference. They should either find a different organization, or try the archipeligo model. There are such thing as irreconcilable value differences.
I agree. I think when that culture was established, the community was missing important concepts about motivated reasoning and truth seeking and chose values that were in fact not optimized for the ultimate goal of creating a community that could solve important problems.
I think it is in fact good to experiment with the norms you’re talking about from the original site, but I think many of those norms originally caused the site to decline and people to go elsewhere. Given my current mental models, I predict a site that uses those norms to make less intellectual progress than a similar site using my norms although I expect you to have the opposite intuition. As I stated in the introduction, the goal of this post was simply to make sure that those mental models were in discourse.
Re your dialogue: The main thing that I got from it was that you think a lot of the arguments in the OP are motivated reasoning and will lead to bad incentives. I also got that this is a subject you care a lot about.
Can you be more specific? Can you name three specific concepts about motivated reasoning and truthseeking that you know, but Sequences-era Overcoming Bias/Less Wrong didn’t?
I mean, that’s one hypothesis. In contrast, my model has been that communities congregate around predictable sources of high-quality writing, and people who can produce high-quality content in high volume are very rare. Thus, once Eliezer Yudkowsky stopped being active, and Yvain a.k.a. the immortal Scott Alexander moved to Slate Star Codex (in part so that he could write about politics, which we’ve traditionally avoided), all the “intellectual energy” followed Scott to SSC.
Can you think of any testable predictions (or retrodictions) that would distinguish my model from your model?
Yes. Thanks for listening.
Here are a few:
The importance of creating a culture that develops Kegan 5 leaders that can take over for the current leaders and help meaningfully change the values as the context changes, in a way that doesn’t simply cause organizations to value drift along with the current broader culture.
How ignoring or not attending for people’s needs creates incentives for motivated reasoning, and how to create spaces that get rid of those incentives WITHOUT being hijacked by whoever screams the loudest.
The importance of cultural tradition and ritual in embedding concepts in augmenting the teaching and telling people what concepts are important.
No because I think that our models are compatible. My model is about how to attract, retain, and develop people with high potential or skill that are in alignment your community’s values, and your model says that not retaining, attracting, or developing people that matched our communities values and had high writing skill is what caused it to fail.
If you can give a specific model of why LW1 failed to attract, retain, and develop high quality writers, then I think there’s a better space for comparison. Perhaps you can also point out some testable predictions that each of our models would make.
First, I want to state that I agree with this model. However, I also want to note that the SSC comments section tend to have fairly low-quality discussion (in comparison to the OB/LW 1.0 heyday), and I’m not sure why this is; candidate hypotheses include that Scott’s explicit politics attracted people with lower epistemic standards, or that the lack of an explicit karma system allowed low-quality discussion to persist (but I don’t think OB had an explicit karma system either?).
Overall, I’m unsure as to what kind of norms/technology maintains high-quality discussion (as opposed to just the presence of discussion in general), and it’s plausible to me that the two may actually be somewhat mutually exclusive (in the sense that norms/technology designed to promote the volume of high-quality discussion may in fact reduce the volume of discussion in general). It’s not clear to me how this tradeoff should be balanced.
(I sometimes think that I might be well-positioned to fill the market niche that Scott occupied in 2014, but no longer can due to his being extortable (“As I became more careful in my own writings [...]”) in a way that I have been trained not to be. But I would need to learn to write faster.)
One thing is that I think early OBNYC and LW just actually had a lot of chaff comments too. I think people disproportionately remember the great parts.
I’m not confident that it does. I perhaps expect people doing this using the native architecture to feel like they’ve found a reasonable answer. But I would expect them to actually be prioritising their own feelings, in most cases. (Though some people will underweight their own feelings. And perhaps some people will get it right.)
Perhaps they will get close enough for the answer to still count as “reasonable”?
If someone attempts to give equal weight to their own needs, the meds of their interlocutor, and the needs of the forum as a whole—how do we know whether they’ve got a reasonable answer? Does that just have to be left to moderator discretion, or?
Yes basically, but if the forum were to take on this direction, the idea would be to have enough case examples/explanations from the moderators about WHY they made that discretion to calibrate people’s reasonable answers. See also this response to Zach which goes more into details about the systems in place to calibrate people’s reasonable answers.
I’m rather confused about what you mean by ‘safe’. I thought I knew what the word meant, but the way you (and some others) are using it perplexes me. Could you explain how to interpret this notion of “safety”?
For instance, this part:
[emphasis mine]
What do the bolded uses of ‘safe’ mean?
Is it the same meaning as the other uses of ‘safe’ in your comment? If not, what other meanings are in use, in which parts of the comment?
I think it’s best defined by its’ antonym. Unsafety, in this context, would mean anything that triggers a defensive or reactive reaction. Just like how bodily unsafety triggers fear, agression, etc, there are psychological equivalents that trigger the same reaction.
Safety is when a particualr circumstance doesn’t trigger that reaction, OR alternatively there could be a meta safety (AKA, having that reaction doesn’t trigger that reaction, because it’s ok).
I think your bolded definitions of safe would actually be served by changing to the word allowed, which for many people correlates quite closely with their feeling of safety.
I think the question of “what is safety?” is a really good one. I’ll write up some thoughts here both for this thread, but also to be to refer to generally (hence a bit more length).
I’m not a fan of that definition. It’s equating “feelings of safety” with “actual safety”
It’s defining safety as the absence the response to perceived unsafety. It feels equivalent to saying “sickness is the thing your immune system fights, and health is the absence of your immune system being triggered to fight something.” Which is very approximately true, but breaks down when you consider autoimmune disorders. With those, it’s the mistaken perception of attack which is the very problem.
This definition can also put a lot of the power in the hands of those who are having a reaction. If we all agree that our conversation must be safe, and that any individual can declare it unsafe because they are having a reaction, this gives a lot power to individuals to force attention on the question of safety (and I fear too asymmetrically with others being blamed for causing the feelings of uncertainty).
-----
So here’s the alternative positive account of “safety” I would give:
One *is* safe if one is unlikely to be harmed; one *feels* if they believe (S1 and/or S2) if they believe they won’t be harmed.
This accords with the standard use of safety, e.g. safety goggles, safety precautions, safe neighborhood, etc.
In conversation, one can be “harmed socially”, e.g. excluded from the group, being “punished” by the group, being made to look bad or stupid (with consequences on how they are treated), having someone act hostilely or aggressive to them (which is a risk of strong negative experience even if they S2 believe it won’t come to any physical or lasting harm), etc. (this is not a carefully developed or complete list).
So in conversation and social spaces, safety equates to not being likely to be harmed in the above ways.
Much the same defenses that activate when feeling under physical threat also come online when feeling under social threat (for indeed, both can be very risky to a human). These are physiological states, fight or flight, etc. How adaptative these are in the modern age . . . more than 0, less than 1 . . .? Having these responses indicates that some part of your mind perceives threat, the question being whether it’s calibrated.
On the question of space: a space can be perceived to have or lower risk of harm to individual (safety) and also higher or lower assessments of risk of harm related to taking specific actions, e.g. saying certain things.
---
With this definition, we can separately evaluate the questions of:
1) Are people actually safe vs likely to be harmed in various ways?
2) Are the harms people worried actually legitimate harms to be worried about?
3) Are people correct to be afraid of being harmed, that is, to feel unsafe?
4) Who should be taking action to cause people to feel unsafe? How is responsibility distributed between the individual and the group?
5) How much should the group/community worry about a) actual safety, and b) perceived safety?
I’m interested in how different people answer these questions generally and in the context of LessWrong.
> I’m not a fan of that definition. It’s equating “feelings of safety” with “actual safety”
I agree with this, but it’s quite a mouthful to deal with. And I think “feelings of safety” are actually more important for truthseeking and creating a product—they’re the things that produce defensiveness, motivated reasoning, etc.
This seems rightish- but off in really important ways that I can’t articulate. It’s putting the emphasis on the wrong things and “collective responsiblity” is not an idea I like at all. I think I’d put my stance as something like “feeling unsafe is a major driver of what people say and do, and good cultures provide space to process and deal with those feelings of unsafety”
Note that this issue is explicitly addressed in the original dialogue. If someones feelings are hurting the discourse, they need to take responsibility for that just as much as I need to take responsibility for hurting their feelings. No one is agreeing that all conversations must be safe for all people, but simply that taking into account when people feel unsafe is important.
Yeah, but there’s a really big difference! You can’t give up that precision.
Nods. Also agree that “collective responsibility” is not the most helpful concept to talk about.
Indeed, the fact people can say “”It feels like your need for safety is getting in the way of truth-seeking”is crucial for it to have any chance.
My expectation based on related real-life experience though, is that if making your need for safety is an option, there will people who abuse this and use it to suck up a lot of time and attention. That technically someone could deny their claim and move on, but this will happen much later than optimal and in the meantime everyone’s attention has been sucked into a great drama. Attempts to say “your safety is disrupting truth-seeking” are accused as being attempts to oppress someone, etc.
This is all imagining how it would go with typical humans. I’m guessing you’re imagining better-than-typical people in your org who won’t have the same failure mode, so maybe it’ll be fine. I’m mostly anchored how I expect that approach to go if applied to most humans I’ve known (especially those really into caring about feelings and who’d be likely to sign up for it).
I think mr-hire thinks the important success condition is that people feel safe and that it’s important to design the space towards this goal, with something of a collective responsibility for the feelings of safety of each individual.
I think Said things that individuals bear full responsibility their feelings of safety, and that it’s actively harmful to make these something the group space has to worry about. I think Said might even believe that “social safety” isn’t even important for the space, i.e., it’s fine if people actually are attacked in social ways, e.g. reputationally harm, caused to be punished by the group, made to experience negative feelings due to aggression from others.
----
If I had to choose between my model of mr-hire’s preferred space and my model of Said’s preferred space, I think I would actually choose Said’s. (Though I might not be correctly characterizing either—I wanted to state my prediction before I asked to test how successfully modeling other’s views).
When it comes to truth seeking, I’d rather err on the side of people getting harmed a bit and having to do a bunch of work to “steel” themselves against the “harsh” environment, then give individuals such a powerful tool (the space being responsible for their perception of being harmed) to disrupt and interfere with discourse. I know that’s not the intended result, but it seems too ripe for abuse to give feelings and needs the primacy I think is being given in the OP scenario. Something like an unachievable utopia: it sounds good, but I am very doubtful it can be done and also be a truth-seeking space.
[Also Said, I had a dream last night that I met you in Central Park, NY. I don’t know what you look or sound like in person, but I enjoyed meeting my dream version of you.]
Well, this is certainly not an egregious strawman by any stretch of the imagination—it’s a reasonable first approximation, really—but I would prefer to be somewhat more precise/nuanced. I would say this:
Individuals bear full responsibility for having their feelings (of safety, yes, and any other relevant propositional attitudes) match the reality as it in fact (objectively/intersubjectively verifiably) presents itself to them.[1]
This, essentially, transforms complaints of “feeling unsafe” into complaints of “being unsafe”; and that is something that we (whoever it is who constitute the “we” in any given case) can consider, and judge. If you’re actually made unsafe by some circumstance, well, maybe we want to do something about that, or prevent it. (Or maybe we don’t, of course. Likely it would depend on the details!) If you’re perfectly safe but you feel unsafe… that’s your own business; deal with it yourself![2]
The relevant questions, again, are about truth and justice. Is it acceptable for people to be reputationally harmed? Well, how is this happening? Certainly libel is not acceptable. Revealing private information about someone (e.g., about their sexual preferences) is not acceptable. Plenty of other things that might cause reputational harm aren’t acceptable. But if you reveal that I, let us say, falsified scientific data (and if this actually is the case), great reputational harm will be done to me; and this is entirely proper. The fact of the harm itself, in other words, is not dispositive.
Similarly for punishment—punishment is proper if it is just, improper otherwise.
As far as “negative feelings” go… “aggression” is a loaded word; what do you mean by it? Suppose that we are having an in-person debate, and you physically assault me; this is “aggression” that would, no doubt, make me “experience negative feelings”; it would also, obviously, be utterly unacceptable behavior. On the other hand, if you did nothing of the sort, but instead made some cutting remark, in which you subtly impugned my intelligence and good taste—is that “aggression”? Or what if you simply said “Said, you’re completely wrong about this, and mistaken in every particular”… aggression? Or not? I might “experience negative feelings” in each of these cases! But the question of whether any of these behaviors are acceptable, or not, does not hinge primarily on whether they could conceivably be described, in some sense, as “aggression”.
In short… when it comes to deciding what is good and what is bad—as with so many other things—precision is everything.
On this, we entirely agree. (And I would add that it is not simply ripe for abuse; it is, in fact, abused, and rampantly, in all cases I have seen.)
Central Park is certainly a pleasant place to meet anyone! I can only hope that, should we ever meet in fact, I live up to the standards set by my dream self…
“reality as it in fact (objectively/intersubjectively verifiably) presents itself to them”: By this somewhat convoluted turn of phrase I mean simply that it’s conceivable for someone to be deceived—made to perceive the facts erroneously, through adversarial action—in which case it would, obviously, be unreasonable to say that it’s entirely the victim’s responsibility to have their feelings about reality match actual reality instead of reality as they are able to discern it; nevertheless this is not license to say “well, this is what the reality feels like to me”, because “what should you reasonably conclude is the reality, given the facts that, as we can all see, are available to you” is something that may be determined and agreed upon, and in no sense is an individual incorrigible on that question.
Which, of course, does not mean that “what’s the best technique for dealing with feeling unsafe when you’re actually safe” isn’t a topic that the group might discuss.
Thanks for the precise and nuanced write-up, and for not objecting to my crude attempt to characterize your position.
Nothing in your views described here strikes me as gravely mistaken, it seems like a sensible norm set. I suspect that many of our disagreements appear once we attempt be precise around acceptable and not acceptable behaviors and how they are handled.
I agree that “aggression” is fuzzy and that simply causing negative emotions is certainly not the criteria by which to judge the acceptability of behavior. I used those terms to indicate/gesture rather than define.
I have a draft, Three ways to upset people with your speech, which attempts to differentiate between importantly different cases. I find myself looking forward to your comments on it once I finally publish it. I don’t think I would have said that a week ago, and I think it’s largely feeling safer with you, which is in turn the result of greater familiarity (I’ve never been active in the LW comments as much as in the last few weeks). I’m more calibrated about the significance of your words now, the degree of malice behind them (possibly not that much?), and even the defensible positions underlying them. I’ve also updated that it’s possible to have a pleasant and valuable exchange.
(I do not say these things because I wish to malign you with my prior beliefs about you, but because I think they’re useful and relevant information.)
Your warm response to my mentioning dream-meeting you made me feel warm (also learning your Myers Briggs type).
(Okay, now please forgive me for using all the above as part of an “argument”; I mean it all genuinely, but it seems to be a very concrete applied way to discuss topics that have been in the air of late.)
This gets us into some tricky questions I can place in your framework. I think it will take us (+ all the others) a fair bit of conversation to answer, but I’ll mention them here now to at least raise them. (Possibly just saying this because I’m away this week and plan not to be online much.)
My updates on you (if correct) suggest that largely Said’s comments do not threaten me much and I shouldn’t feel negative feelings as a result. Much of this is just how Said talks, and he’s still interested in honest debate, not just shutting you down with hostile talk. But my question is the “reality as it presents itself to me” you mentioned. The reality might be that Said is safe, but was I, given my priors and evidence available to me before, wrong to be afraid before I gained more information about how to interpret Said?
(Maybe I was, but this is not obvious.)
Is the acceptability of behavior determined by what the recipient reasonably could have believed (as judged by . . . ?) or by the actual reality. Or there are three possibilities even: 1) what I could have reasonably believed was the significance of your actions, 2) what you could have reasonably believed was the significance of your actions, 3) what the actual significance of your actions were (if this can even be defined sensibly).
It does seem somewhat unfair if the acceptability of your behavior is impacted by what I can reasonably believe. It also seems somewhat unfair that I should experience attack because I reasonably lacked information.
How do we handle all this? I don’t definitively know. Judging what is acceptable/reasonable/fair and how all different perspectives add up . . . it’s a mess that I don’t think gets better even with more attempt at precision. I mostly want to avoid having to judge.
This is in large part what intuitively pushes me towards wanting people to be proactive in avoiding misinterpretations and miscalibrations of other’s intent—so we don’t have to judge who was at fault. I want people to give people enough info that they correctly know even when I’m harsh, I still want them to feel safe. Mostly applies to people who don’t know me well. Once the evidence has accrued and you’re calibrated on what things mean, you require little “padding” (this is my version of Combat culture essentially), but you’ve got to accrue that evidence and establish the significance of actions with others first.
--
Phew, like everything else, that was longer than expected. I should really start expected everything to be long.
Curious if this provides any more clarity on my position (even if it’s not persuasive) and curious where you disagree with this treatment.
Here’s the couple of thousand words that fell out when I attempted to write up my thoughts re safety and community norms.
Link seems broken
Thanks, fixed! It’s a little bit repetitive with everything else I’ve written lately, but maybe I’m getting it clearer with each iteration.
I ended up having thoughts here that grew beyond the context (how to think about this feels related to how to think about depleted willpower). Wrote a shortform post.
My current best guess is that you’d get the best results (measured roughly “useful ideas generated and non-useful ideas pruned”) from a collection of norms where
a) people need to take responsibility for their own feelings of fear, and there is help/guidelines on how to go about that if you’re not very good at it yet, and
b) people also take responsibility for learning to the social and writing skills to avoid particularly obvious failure modes.
i.e. “I’m feeling defensive” shouldn’t be a get out of jail free card. (in particular, any request that someone change their communication behavior should come with a corresponding costly signal that you are working on improving your ability to listen while triggered)
And while I think I’ve believed this for a couple weeks, I don’t think I was doing the work to actually embody it, and I think that’s been a mistake I’ve been making.
I’m holding the frame you wrote on your shortform feed re defensiveness for a bit to see how I feel about it.
I’ve been trying to use the phrase ‘feeling of safety’ when it comes up but it has the unfortunate property that ‘aspiring rationalist’ had, where there isn’t a stable equilibrium where people reliably say the whole phrase.
I hereby proclaim that “feelings of safety” be shortened to “fafety.” The domain of worrying about fafety is now “fafety concerns.”
Problem solved. All in a day’s work.
Strong upvote