Thanks for writing this! I’d been putting something together, but this is much more thorough. Here are the parts of my draft that I think still add something:
I’m interested in two overlapping questions:
Should Ben have delayed to evaluate NL’s evidence?
Was Nonlinear wrong to threaten to sue?
While I’ve previously advocated giving friendly organizations a chance to review criticism and prepare a response in advance, primarily as a question of politeness, that’s not the issue here. As I commented on the original post, the norm I’ve been pushing is only intended for cases where you have a neutral or better relationship with the organization, and not situations like this one where there are allegations of mistreatment or you don’t trust them to behave cooperatively. The question here instead is, how do you ensure the accusations you’re signal-boosting are true?
Here’s my understanding of the timeline of ‘adversarial’ fact checking before publication: timeline. Three key bits:
LC first shared the draft 21hr before posting, which included additional accusations
NL responded to both by asking for a week to gather evidence that they claimed would change LC’s mind, and only threatened to sue when LC insisted on going ahead without hearing their side of the story.
Overall, it does look to me like Ben should have waited. That he was still learning his post had additional false accusations right up to publication, including one so close to publication that he didn’t have time to address it, meant he should have expected that if he didn’t delay it would go to publication containing additional false claims. And Ben seems to have understood this, and told NL two days before that “I do expect you’ll be able to show a bunch of the things you said” and “I did update from things you shared that Alice’s reports are less reliable than I had thought”. Additionally, three days (some of which NL was traveling for) is not long to rebut such a long list of accusations, and some of the accusations were shared with NL less than a day in advance of publishing.
Given that I think it’s clear Ben should have given NL some time to assemble conflicting evidence, do I also need to conclude that it was ok for NL to threaten to sue if he didn’t? That is something I really don’t want to do: I’m not a fan of how strict defamation law is, and I think it’s really valuable that it’s almost never used in our community. If we instead had a culture where everyone is being super careful never to say anything that they would lose a suit over I think we’d see a tremendous chilling effect where many true and important things would never come out. This is especially important around allegations of abusive behavior, where proving the truth of anything can be quite difficult and abusers have often taken advantage of this.
On the other hand, defamation law at its best deters people from saying false things, including fact checking before republishing others’ claims, especially in cases where false statements are likely to have strong personal and professional impacts. NL threatening a suit here seems to me like the law used correctly even given the tighter scope I’d like to see for defamation law, and while it’s still not something I would do and I’d be sad if it became common in our community, I don’t think NL was actually wrong to do it.
Note that I’m not making an overall “Nonlinear: yes or no” judgement here; please don’t interpret this post as a view on whether NL mistreated their employees or on how trustworthy their former employees are. I’ve only been looking at the two questions above: should Ben have waited for additional evidence, and was it an unreasonable escalation for NL to threaten to sue.
A few thoughts about how future posts like LC’s could go better:
Be clear about voice. Ben’s post often clarifies that he’s relaying information (“Alice reported that”) but at other times reports Alice or Chloe’s perspective as his own (“nobody in the house was willing to go out and get her vegan food, so she barely ate for 2 days”). The latter would be warranted if Ben had validated the claims he’s relaying, but my understanding is this was instead just a bit sloppy.
If you have your facts straight you don’t have to give the people you’re writing about a chance to look over a draft and point out errors, and in cases where, for example, people are telling the story of abuse that happened to them that’s often a reasonable decision. On the other hand, especially if you’re reporting on a situation you weren’t a part of, I think you’re really very likely to have some mistakes. Sharing your post’s claims section (you don’t have to share parts of the post that don’t make new claims) for review is really hard to beat as a way of avoiding publishing false claims.
How much time to give for assembling evidence depends on how many claims there are, and I think you can do some prioritization. You can ask if they can provide evidence for some of the places they think you’re most clearly wrong, and if they can’t do a good job with this given a few days I think you’re justified in assuming their claims of counter-evidence are bluster.
When giving time to fact check and gather evidence, get agreement on deadlines up front so things don’t get pushed back repeatedly: “I’ll review anything you get me before EOD Thursday; does that work?” While you should sort out all your claims before starting this process, if you want to add new claims after then you need to push the deadline out to give them more time to respond.
I really like Michael Plant’s comment on a previous post about issues with an EA org (which @Habryka also worked on), responding to someone who thought the post was “charitable and systematic in excess of reasonable caution” and objected to the “fully comprehensive nature of the post and the painstaking lengths it goes to separate out definitely valid issues from potentially invalid ones”:
I take your point as “aren’t we being too nice to this guy?” but I actually really like the approach taken here, which seems extremely fair-minded and diligent. My suspicion is this sort of stuff is long-term really valuable because it establishes good norms for something that will likely recur in future. I’d be much more inclined to act with honesty if I believed people would do an extremely thorough public investigation into everything I’d said, rather than just calling me names and walking away.
If you think that NL was overall in the right then of course LC should have been more careful, but even if you think NL is run by horrible people I think you still should come to that conclusion. This combination of posts and overwhelming amounts of conflicting screenshots and stories seems to have left most of the community not knowing what to think. A post from LC that led with airtight accusations and then included supplementary material for fuzzier issues would have been much more effective towards its goals, enough so that I think more hours spent in ‘adversarial fact checking’, resolving conflicting claims, would have been far more effective per hour than the average for this project.
I almost always post immediately after writing things, since that’s how my motivation works, but in this case I sent my post out for review. I got some good feedback from several people, but now that the top-level post has opened the conversation about appropriate process in situations like these I think talking in public makes more sense.
Someone asked me privately whether it would change my overall view if I learned that none of the claims that NL and Spencer offered evidence against during the 3 days (and/or 21hr) they had the list were false. I don’t think it would, unless I also learned that LC was rightly completely confident of this at the time, which is quite a high bar given how little time there had been for NL to present their side.
Another response to a private question: is there any way to run an investigation like this if you absolutely can’t share the claims with the accused ahead of time and wait for them to give counter-evidence?
I do think you can make a post that (a) compiles public evidence, (b) presents new evidence, and/or (c) presents your personal experience while getting your facts sufficiently correct that a defamation lawsuit would be an unreasonable escalation and unlikely to succeed. When signal-boosting someone else’s accusations, however, especially if any of it involves taking their word for things, this is far harder and I expect almost always impossible.
If you have a hard constraint that you can’t run something by the accused before publishing, I think you have two main options:
A. Support the accuser in getting their own story out: helping them draft it, gathering supporting evidence, helping them figure out which claims to lead with and which to drop, proofreading, emotional support, etc.
B. Write something that contains only what you can fully defend. “Here are screenshots Pat shared with us of a conversation with Org that we found troubling.” If Pat is willing to go as far as forging evidence, and is good at it, then you’re just screwed, but I’m assuming you’re willing to take the risk that they’re not doing that.
Crossposted from the EA Forum
Thanks for writing this! I’d been putting something together, but this is much more thorough. Here are the parts of my draft that I think still add something:
I’m interested in two overlapping questions:
Should Ben have delayed to evaluate NL’s evidence?
Was Nonlinear wrong to threaten to sue?
While I’ve previously advocated giving friendly organizations a chance to review criticism and prepare a response in advance, primarily as a question of politeness, that’s not the issue here. As I commented on the original post, the norm I’ve been pushing is only intended for cases where you have a neutral or better relationship with the organization, and not situations like this one where there are allegations of mistreatment or you don’t trust them to behave cooperatively. The question here instead is, how do you ensure the accusations you’re signal-boosting are true?
Here’s my understanding of the timeline of ‘adversarial’ fact checking before publication: timeline. Three key bits:
LC first shared the overview of claims 3d before posting.
LC first shared the draft 21hr before posting, which included additional accusations
NL responded to both by asking for a week to gather evidence that they claimed would change LC’s mind, and only threatened to sue when LC insisted on going ahead without hearing their side of the story.
Overall, it does look to me like Ben should have waited. That he was still learning his post had additional false accusations right up to publication, including one so close to publication that he didn’t have time to address it, meant he should have expected that if he didn’t delay it would go to publication containing additional false claims. And Ben seems to have understood this, and told NL two days before that “I do expect you’ll be able to show a bunch of the things you said” and “I did update from things you shared that Alice’s reports are less reliable than I had thought”. Additionally, three days (some of which NL was traveling for) is not long to rebut such a long list of accusations, and some of the accusations were shared with NL less than a day in advance of publishing.
Given that I think it’s clear Ben should have given NL some time to assemble conflicting evidence, do I also need to conclude that it was ok for NL to threaten to sue if he didn’t? That is something I really don’t want to do: I’m not a fan of how strict defamation law is, and I think it’s really valuable that it’s almost never used in our community. If we instead had a culture where everyone is being super careful never to say anything that they would lose a suit over I think we’d see a tremendous chilling effect where many true and important things would never come out. This is especially important around allegations of abusive behavior, where proving the truth of anything can be quite difficult and abusers have often taken advantage of this.
On the other hand, defamation law at its best deters people from saying false things, including fact checking before republishing others’ claims, especially in cases where false statements are likely to have strong personal and professional impacts. NL threatening a suit here seems to me like the law used correctly even given the tighter scope I’d like to see for defamation law, and while it’s still not something I would do and I’d be sad if it became common in our community, I don’t think NL was actually wrong to do it.
Note that I’m not making an overall “Nonlinear: yes or no” judgement here; please don’t interpret this post as a view on whether NL mistreated their employees or on how trustworthy their former employees are. I’ve only been looking at the two questions above: should Ben have waited for additional evidence, and was it an unreasonable escalation for NL to threaten to sue.
A few thoughts about how future posts like LC’s could go better:
Be clear about voice. Ben’s post often clarifies that he’s relaying information (“Alice reported that”) but at other times reports Alice or Chloe’s perspective as his own (“nobody in the house was willing to go out and get her vegan food, so she barely ate for 2 days”). The latter would be warranted if Ben had validated the claims he’s relaying, but my understanding is this was instead just a bit sloppy.
If you have your facts straight you don’t have to give the people you’re writing about a chance to look over a draft and point out errors, and in cases where, for example, people are telling the story of abuse that happened to them that’s often a reasonable decision. On the other hand, especially if you’re reporting on a situation you weren’t a part of, I think you’re really very likely to have some mistakes. Sharing your post’s claims section (you don’t have to share parts of the post that don’t make new claims) for review is really hard to beat as a way of avoiding publishing false claims.
How much time to give for assembling evidence depends on how many claims there are, and I think you can do some prioritization. You can ask if they can provide evidence for some of the places they think you’re most clearly wrong, and if they can’t do a good job with this given a few days I think you’re justified in assuming their claims of counter-evidence are bluster.
When giving time to fact check and gather evidence, get agreement on deadlines up front so things don’t get pushed back repeatedly: “I’ll review anything you get me before EOD Thursday; does that work?” While you should sort out all your claims before starting this process, if you want to add new claims after then you need to push the deadline out to give them more time to respond.
I really like Michael Plant’s comment on a previous post about issues with an EA org (which @Habryka also worked on), responding to someone who thought the post was “charitable and systematic in excess of reasonable caution” and objected to the “fully comprehensive nature of the post and the painstaking lengths it goes to separate out definitely valid issues from potentially invalid ones”:
If you think that NL was overall in the right then of course LC should have been more careful, but even if you think NL is run by horrible people I think you still should come to that conclusion. This combination of posts and overwhelming amounts of conflicting screenshots and stories seems to have left most of the community not knowing what to think. A post from LC that led with airtight accusations and then included supplementary material for fuzzier issues would have been much more effective towards its goals, enough so that I think more hours spent in ‘adversarial fact checking’, resolving conflicting claims, would have been far more effective per hour than the average for this project.
I almost always post immediately after writing things, since that’s how my motivation works, but in this case I sent my post out for review. I got some good feedback from several people, but now that the top-level post has opened the conversation about appropriate process in situations like these I think talking in public makes more sense.
Someone asked me privately whether it would change my overall view if I learned that none of the claims that NL and Spencer offered evidence against during the 3 days (and/or 21hr) they had the list were false. I don’t think it would, unless I also learned that LC was rightly completely confident of this at the time, which is quite a high bar given how little time there had been for NL to present their side.
Another response to a private question: is there any way to run an investigation like this if you absolutely can’t share the claims with the accused ahead of time and wait for them to give counter-evidence?
I do think you can make a post that (a) compiles public evidence, (b) presents new evidence, and/or (c) presents your personal experience while getting your facts sufficiently correct that a defamation lawsuit would be an unreasonable escalation and unlikely to succeed. When signal-boosting someone else’s accusations, however, especially if any of it involves taking their word for things, this is far harder and I expect almost always impossible.
If you have a hard constraint that you can’t run something by the accused before publishing, I think you have two main options:
A. Support the accuser in getting their own story out: helping them draft it, gathering supporting evidence, helping them figure out which claims to lead with and which to drop, proofreading, emotional support, etc.
B. Write something that contains only what you can fully defend. “Here are screenshots Pat shared with us of a conversation with Org that we found troubling.” If Pat is willing to go as far as forging evidence, and is good at it, then you’re just screwed, but I’m assuming you’re willing to take the risk that they’re not doing that.