I think the burden of evidence is on the side disagreeing with the intuitions behind this extremely common defensive response
Note also that most groups treat their intuitions about whether or not someone is acting in bad faith as evidence worth taking seriously, and that we’re remarkable in how rarely we tend to allow our bad-faith-detecting intuitions to lead us to reach the positive conclusion that someone is acting in bad faith. Note also that we have a serious problem with not being able to effectively deal with Gleb-like people, sexual predators, etc, and that these sorts of people reliably provoke person-acting-in-bad-faith-intuitions in people with (both) strong and accurate bad-faith-sensing intuitions. (Note that having strong bad-faith-detecting intuitions correlates somewhat with having accurate ones, since having strong intuitions here makes it easier to pay attention to your training data, and thus build better intuitions with time). Anyways, as a community, taking intuitions about when someone’s acting in bad faith more seriously on the margin could help with this.
Now, one problem with this strategy is that many of us are out of practice at using these intuitions! It also doesn’t help that people without accurate bad-faith-detecting intuitions often typical-mind fallacy their way into believing that there aren’t people who have exceptionally accurate bad-faith-detecting intuitions. Sometimes this gets baked into social norms, such that criticism becomes more heavily taxed, partly because people with weak bad-faith-detecting intuitions don’t trust others to direct their criticism at people who are actually acting in bad faith.
Of course, we currently don’t accept person-acting-in-bad-faith-intuitions as useful evidence in the EA/LW community, so people who provoke more of these intuitions are relatively more welcome here than in other groups. Also, for people with both strong and accurate bad-faith-detecting intuitions, being around people who set off their bad-faith-sensing intuitions isn’t fun, so such people feel less welcome here, especially since a form of evidence they’re good at acquiring isn’t socially acknowledged or rewarded, while it is acknowledged and rewarded elsewhere. And when you look around, you see that we in fact don’t have many people with strong and accurate bad-faith-detecting intuitions; having more of these people around would have been a good way to detect Gleb-like folks much earlier than we tend to.
How acceptable bad-faith-detecting intuitions are in decision-making is also highly relevant to the gender balance of our community, but that’s a topic for another post. The tl;dr of it is that, when bad-faith-detecting intuitions are viewed as providing valid evidence, it’s easier to make people who are acting creepy change how they’re acting or leave, since “creepiness” is a non-objective thing that nevertheless has a real, strong impact on who shows up at your events.
Anyhow, I’m incredibly self-interested in pointing all of this out, because I have very strong (and, as of course I will claim, very accurate) bad-faith-detecting intuitions. If people with stronger bad-faith-detecting intuitions are undervalued because our skill at detecting bad actors isn’t recognized, then, well, this implies people should listen to us more. :P
Here on LW Gleb got laughed at almost immediately as he started posting. Did he actually manage to make any inroads into EA/Bay Area communities? I know EA ended up writing a basically “You are not one of us, please go away” post/letter, but it took a while.
Amusingly, one possible explanation is that the people who gave Gleb pushback on here were operating on bad-faith-detecting intuitions—this is supported by the quick reaction time. I’d say that those intuitions were good ones, if they lead to those folks giving Gleb pushback on a quick timescale, and I’d also say that those intuitions shaped healthy norms to the extent that they nudged us towards establishing a quick reality-grounded social feedback loop.
But the people who did give Gleb pushback more frequently framed things in terms other than them having bad-faith-detecting intuitions than you’d have guessed, if they were actually concluding that giving Gleb pushback was worth their time based on their intuitions—they pointed to specific behaviors, and so on, when calling him out. But how many of these people actually decided to give Gleb feedback because they System-2-noticed that he was implementing a specific behavior, and how many of us decided to give Gleb feedback because our bad-faith-detecting intuitions noticed something was up, which led us to fish around for a specific bad behavior that Gleb was doing?
If more of us did the latter, this suggests that we have social incentives in place that reward fishing around and finding specific bad behaviors, but to me, fishing around for bad behaviors (i.e. fishing through data) like this doesn’t seem too much different from p-hacking, except that fishing around for social data is way harder to call people out on. And if our real reasons for reaching the correct conclusion that Gleb needed to get pushback were based in bad-faith-detecting intuitions, and not in System 2 noticing bad behaviors, then maybe providing social allowance for the mechanism that actually led some of us to detect Gleb a bit earlier to do its work on its own in the future, rather than requiring its use to be backed up by evidence of bad behaviors (junk data) that can be both p-hacked by those who want to criticize independently of what was true, or hidden by those with more skill than Gleb, would be a good idea.
At a minimum, being honest with ourselves about what our real reasons are ought to help us understand our minds a bit better.
But how many of these people actually decided to give Gleb feedback because they System-2-noticed that he was implementing a specific behavior, and how many of us decided to give Gleb feedback because our bad-faith-detecting intuitions noticed something was up, which led us to fish around for a specific bad behavior that Gleb was doing?
I don’t know if you can separate it this cleanly. Sometimes you get a smells-funny feeling and then your System 2 goes to investigate. But sometimes—and I think this was the case with Gleb—both System 1 and System 2 look at each other and chorus “Really, dude?” :-)
nod. This does seem like it should be a continuous thing, rather than System 1 solely figuring things out in some cases and System 2 figuring it out alone in others.
I sent a few private notes to him early on about the way I reacted to his posts. This wasn’t a “bad faith” detector ( I don’t actually buy the premise—such a thing is VERY uncommon compared to honest incorrect values and beliefs), this was a pattern match to an overzealous overconfident newbie, possibly with under-developed social skills. You know, just like all of us a few years (or in my case decades) ago.
This all sounds right, but the reasoning behind using the wording of “bad faith” is explained in the second bullet point of this comment.
Tl;dr the module your brain has for detecting things that feel like “bad faith” is good at detecting when someone is acting in ways that cause bad consequences in expectation but don’t feel like “bad faith” to the other person on the inside. If people could learn to correct a subset of these actions by learning, say, common social skills, treating those actions like they’re taken in “bad faith” incentivizes them to learn those skills, which results in you having to live with negative consequences from dealing with that person less. I’d say that this is part of why our minds often read well-intentioned-but-harmful-in-expectation behaviors as “bad faith”; it’s a way of correcting them.
I’d guess the same fraction of people reacted disrespectfully to Gleb in each community (i.e. most but not all). The difference was more that in an EA context, people worried that he would shift money away from EA-aligned charities, but on LW he only wasted peoples’ time.
I’m very glad that you asked this! I think we can come up with some decent heuristics:
If you start out with some sort of inbuilt bad faith detector, try to see when, in retrospect, it’s given you accurate readings, false positives, and false negatives. I catch myself doing this without having planned to on a System 1 level from time to time. It may be possible, if harder, to do this sort of intuition reshaping in response to evidence with System 2. Note that it sometimes takes a long time, and that sometimes you never figure out, whether or not your bad-faith-detecting intuitions were correct.
There’s debate about whether a bad-faith-detecting intuition that fires when someone “has good intentions” but ends up predictably acting in ways that hurt you (especially to their own benefit) is “correct”. My view is that the intuition is correct; defining it as incorrect and then acting in social accordance with it being incorrect incentivizes others to manipulate you by being/becoming good at making themselves believe they have good intentions when they don’t, which is a way of destroying information in itself. Hence why allowing people to get away with too many plausibly deniable things destroys information: if plausible deniability is a socially acceptable defense when it’s obvious someone has hurt you in a way that benefits them, they’ll want to blind themselves to information about how their own brains work. (This is a reason to disagree with many suggestions made in Nate’s post. If treating people like they generally have positive intentions reduces your ability to do collaborative truth-seeking with others on how their minds can fail in ways that let you down—planning fallacy is one example—then maybe it would be helpful to socially disincentivize people from misleading themselves this way by giving them critical feedback, or at least not tearing people down for being ostracizers when they do the same).
Try to evaluate other’s bad faith detectors by the same mechanism as in the first point; if they give lots of correct readings and not many false ones (especially if they share their intuitions with you before it becomes obvious to you whether or not they’re correct), this is some sort of evidence that they have strong and accurate bad-faith-detecting intuitions.
The above requires that you know someone well enough for them to trust you with this data, so a quicker way to evaluate other’s bad-faith-detecting intuitions is to look at who they give feedback to, criticize, praise, etc. If they end up attacking or socially qualifying popular people who are later revealed to have been acting in bad faith, or if they end up praising or supporting ones who are socially suspected of being up to something who are later revealed to have been acting in good faith, these are strong signals of them having accurate bad-faith-detecting intuitions.
Done right, bad-faith-detecting intuitions should let you make testable predictions about who will impose costs or provide benefits to you and your friends/cause; these intuitions become more valuable as you become more accurate at evaluating them. Bad-faith-detecting intuitions might not “taste” like Officially Approved Scientific Evidence,
and we might not respect them much around here, but they should tie back into reality, and be usable to help you make better decisions than you’d been able to make without using them.
Note also that most groups treat their intuitions about whether or not someone is acting in bad faith as evidence worth taking seriously
Of course our intuitions of someone acting in bad faith are evidence that they are; that much is obvious. The relevant question is how strong the correlation is between the intuition vs. actual bad faith. Since you admit quite openly that people vary widely in how sensitive their intuitive ‘bad-faith detectors’ are (this, after all, it what it means to have a ‘strong sense’ of such!), shouldn’t this be of concern for those who would claim that this correlation is very high—quite high enough to be useful on its own?
It’s also important to realize that both Type I (false hit) and Type II (miss) errors are harmful here, hence, as usual in any binary detection setting, specificity is as relevant as sensitivity—and there’s no reason why additional evidence should be discounted; particularly if such evidence is of a factual sort—and as such is likely to be otherwise broadly independent from the output of our intuitive detectors!
Note also that most groups treat their intuitions about whether or not someone is acting in bad faith as evidence worth taking seriously, and that we’re remarkable in how rarely we tend to allow our bad-faith-detecting intuitions to lead us to reach the positive conclusion that someone is acting in bad faith. Note also that we have a serious problem with not being able to effectively deal with Gleb-like people, sexual predators, etc, and that these sorts of people reliably provoke person-acting-in-bad-faith-intuitions in people with (both) strong and accurate bad-faith-sensing intuitions. (Note that having strong bad-faith-detecting intuitions correlates somewhat with having accurate ones, since having strong intuitions here makes it easier to pay attention to your training data, and thus build better intuitions with time). Anyways, as a community, taking intuitions about when someone’s acting in bad faith more seriously on the margin could help with this.
Now, one problem with this strategy is that many of us are out of practice at using these intuitions! It also doesn’t help that people without accurate bad-faith-detecting intuitions often typical-mind fallacy their way into believing that there aren’t people who have exceptionally accurate bad-faith-detecting intuitions. Sometimes this gets baked into social norms, such that criticism becomes more heavily taxed, partly because people with weak bad-faith-detecting intuitions don’t trust others to direct their criticism at people who are actually acting in bad faith.
Of course, we currently don’t accept person-acting-in-bad-faith-intuitions as useful evidence in the EA/LW community, so people who provoke more of these intuitions are relatively more welcome here than in other groups. Also, for people with both strong and accurate bad-faith-detecting intuitions, being around people who set off their bad-faith-sensing intuitions isn’t fun, so such people feel less welcome here, especially since a form of evidence they’re good at acquiring isn’t socially acknowledged or rewarded, while it is acknowledged and rewarded elsewhere. And when you look around, you see that we in fact don’t have many people with strong and accurate bad-faith-detecting intuitions; having more of these people around would have been a good way to detect Gleb-like folks much earlier than we tend to.
How acceptable bad-faith-detecting intuitions are in decision-making is also highly relevant to the gender balance of our community, but that’s a topic for another post. The tl;dr of it is that, when bad-faith-detecting intuitions are viewed as providing valid evidence, it’s easier to make people who are acting creepy change how they’re acting or leave, since “creepiness” is a non-objective thing that nevertheless has a real, strong impact on who shows up at your events.
Anyhow, I’m incredibly self-interested in pointing all of this out, because I have very strong (and, as of course I will claim, very accurate) bad-faith-detecting intuitions. If people with stronger bad-faith-detecting intuitions are undervalued because our skill at detecting bad actors isn’t recognized, then, well, this implies people should listen to us more. :P
Here on LW Gleb got laughed at almost immediately as he started posting. Did he actually manage to make any inroads into EA/Bay Area communities? I know EA ended up writing a basically “You are not one of us, please go away” post/letter, but it took a while.
Good observation.
Amusingly, one possible explanation is that the people who gave Gleb pushback on here were operating on bad-faith-detecting intuitions—this is supported by the quick reaction time. I’d say that those intuitions were good ones, if they lead to those folks giving Gleb pushback on a quick timescale, and I’d also say that those intuitions shaped healthy norms to the extent that they nudged us towards establishing a quick reality-grounded social feedback loop.
But the people who did give Gleb pushback more frequently framed things in terms other than them having bad-faith-detecting intuitions than you’d have guessed, if they were actually concluding that giving Gleb pushback was worth their time based on their intuitions—they pointed to specific behaviors, and so on, when calling him out. But how many of these people actually decided to give Gleb feedback because they System-2-noticed that he was implementing a specific behavior, and how many of us decided to give Gleb feedback because our bad-faith-detecting intuitions noticed something was up, which led us to fish around for a specific bad behavior that Gleb was doing?
If more of us did the latter, this suggests that we have social incentives in place that reward fishing around and finding specific bad behaviors, but to me, fishing around for bad behaviors (i.e. fishing through data) like this doesn’t seem too much different from p-hacking, except that fishing around for social data is way harder to call people out on. And if our real reasons for reaching the correct conclusion that Gleb needed to get pushback were based in bad-faith-detecting intuitions, and not in System 2 noticing bad behaviors, then maybe providing social allowance for the mechanism that actually led some of us to detect Gleb a bit earlier to do its work on its own in the future, rather than requiring its use to be backed up by evidence of bad behaviors (junk data) that can be both p-hacked by those who want to criticize independently of what was true, or hidden by those with more skill than Gleb, would be a good idea.
At a minimum, being honest with ourselves about what our real reasons are ought to help us understand our minds a bit better.
I don’t know if you can separate it this cleanly. Sometimes you get a smells-funny feeling and then your System 2 goes to investigate. But sometimes—and I think this was the case with Gleb—both System 1 and System 2 look at each other and chorus “Really, dude?” :-)
nod. This does seem like it should be a continuous thing, rather than System 1 solely figuring things out in some cases and System 2 figuring it out alone in others.
I sent a few private notes to him early on about the way I reacted to his posts. This wasn’t a “bad faith” detector ( I don’t actually buy the premise—such a thing is VERY uncommon compared to honest incorrect values and beliefs), this was a pattern match to an overzealous overconfident newbie, possibly with under-developed social skills. You know, just like all of us a few years (or in my case decades) ago.
This all sounds right, but the reasoning behind using the wording of “bad faith” is explained in the second bullet point of this comment.
Tl;dr the module your brain has for detecting things that feel like “bad faith” is good at detecting when someone is acting in ways that cause bad consequences in expectation but don’t feel like “bad faith” to the other person on the inside. If people could learn to correct a subset of these actions by learning, say, common social skills, treating those actions like they’re taken in “bad faith” incentivizes them to learn those skills, which results in you having to live with negative consequences from dealing with that person less. I’d say that this is part of why our minds often read well-intentioned-but-harmful-in-expectation behaviors as “bad faith”; it’s a way of correcting them.
I’d guess the same fraction of people reacted disrespectfully to Gleb in each community (i.e. most but not all). The difference was more that in an EA context, people worried that he would shift money away from EA-aligned charities, but on LW he only wasted peoples’ time.
http://lesswrong.com/lw/ou5/against_responsibility/dqfj
Agree in theory, but, lacking an effective bad faith detector myself, how do I know whose intuitions to trust? :(
I’m very glad that you asked this! I think we can come up with some decent heuristics:
If you start out with some sort of inbuilt bad faith detector, try to see when, in retrospect, it’s given you accurate readings, false positives, and false negatives. I catch myself doing this without having planned to on a System 1 level from time to time. It may be possible, if harder, to do this sort of intuition reshaping in response to evidence with System 2. Note that it sometimes takes a long time, and that sometimes you never figure out, whether or not your bad-faith-detecting intuitions were correct.
There’s debate about whether a bad-faith-detecting intuition that fires when someone “has good intentions” but ends up predictably acting in ways that hurt you (especially to their own benefit) is “correct”. My view is that the intuition is correct; defining it as incorrect and then acting in social accordance with it being incorrect incentivizes others to manipulate you by being/becoming good at making themselves believe they have good intentions when they don’t, which is a way of destroying information in itself. Hence why allowing people to get away with too many plausibly deniable things destroys information: if plausible deniability is a socially acceptable defense when it’s obvious someone has hurt you in a way that benefits them, they’ll want to blind themselves to information about how their own brains work. (This is a reason to disagree with many suggestions made in Nate’s post. If treating people like they generally have positive intentions reduces your ability to do collaborative truth-seeking with others on how their minds can fail in ways that let you down—planning fallacy is one example—then maybe it would be helpful to socially disincentivize people from misleading themselves this way by giving them critical feedback, or at least not tearing people down for being ostracizers when they do the same).
Try to evaluate other’s bad faith detectors by the same mechanism as in the first point; if they give lots of correct readings and not many false ones (especially if they share their intuitions with you before it becomes obvious to you whether or not they’re correct), this is some sort of evidence that they have strong and accurate bad-faith-detecting intuitions.
The above requires that you know someone well enough for them to trust you with this data, so a quicker way to evaluate other’s bad-faith-detecting intuitions is to look at who they give feedback to, criticize, praise, etc. If they end up attacking or socially qualifying popular people who are later revealed to have been acting in bad faith, or if they end up praising or supporting ones who are socially suspected of being up to something who are later revealed to have been acting in good faith, these are strong signals of them having accurate bad-faith-detecting intuitions.
Done right, bad-faith-detecting intuitions should let you make testable predictions about who will impose costs or provide benefits to you and your friends/cause; these intuitions become more valuable as you become more accurate at evaluating them. Bad-faith-detecting intuitions might not “taste” like Officially Approved Scientific Evidence, and we might not respect them much around here, but they should tie back into reality, and be usable to help you make better decisions than you’d been able to make without using them.
Of course our intuitions of someone acting in bad faith are evidence that they are; that much is obvious. The relevant question is how strong the correlation is between the intuition vs. actual bad faith. Since you admit quite openly that people vary widely in how sensitive their intuitive ‘bad-faith detectors’ are (this, after all, it what it means to have a ‘strong sense’ of such!), shouldn’t this be of concern for those who would claim that this correlation is very high—quite high enough to be useful on its own?
It’s also important to realize that both Type I (false hit) and Type II (miss) errors are harmful here, hence, as usual in any binary detection setting, specificity is as relevant as sensitivity—and there’s no reason why additional evidence should be discounted; particularly if such evidence is of a factual sort—and as such is likely to be otherwise broadly independent from the output of our intuitive detectors!