Thanks for the outside perspective. If you’re willing to go into more detail, I’m interested in a more detailed account from you on both what academia’s safeguards are and (per gwillen’s comment) where do you think academia’s safeguards fall short and how that can be fixed.
This is decision-relevant to me as I work in a research organization outside of academia (though not working on AI risk specifically), and I would like us to both be more productive than typical in academia and have better safeguards against abuse.
If it helps, we have about 15 researchers now, we’re entirely remote, and we hire typically from people who just finished their PhDs or have roughly equivalent research experience, although research interns/fellows are noticeably younger (maybe right after undergrad is the median).
Sure. I’m really glad to hear. This is not my community, but you did explicitly ask.
This is just off the top of my head, and I don’t mean it to be a final complete and correct list. It’s just to give you a sense of some things I’ve encountered, and to help you and your org think about how to empower people and help them flourish. Academia uses a lot of these to avoid the geek-MOP-sociopath cycle.
I’m assuming your institution wants to follow an academic model, including teaching, mentorship, hiearchical student-teacher relationships, etc.
An open question is when you have a duty of care. My rule of thumb is (1) when you or the org is explicitly saying “I’m your teacher”, “I’m your mentor”; (2) when you feel a power imbalance with someone because this relationship has arisen implicitly; (3) when someone is soliciting this role from you, whether you want it or not.
If you’re a business making money, that’s quite different, just say “we’re going to use your body and mind to make money” and you’ve probably gotten your informed consent. :)
* Detection
1. Abuse is non-Gaussian. A small number of people may experience a great deal, while the majority see nothing wrong. That means that occasional random sampling is not going to identify problems. There are a lot of comments here saying “XYZ org (etc) was great, I saw nothing bad” — this is not a good signal.
2. Women and people from marginalized groups are at much higher risk. They’re less able to trust a random stranger, and they’re also less able to appeal to social norms or law enforcement. They also are at higher risk if they do report.
Somebody in the comments said that many of the people reporting abuse are trans, and “trans people suffer from mental illness more”, so maybe they’re just crazy and everything was actually pretty OK.
Hopefully this reasoning looks as crazy to you as it does to me; in the 1970s people would have said the same about gay people, but now we realize that a lot of that was due to homophobia (etc), and a lot of it was due to the fact that gay people, being marginalized, made soft targets for manipulation, blackmail, etc.
3. Always take reports seriously, even if “the person seems weird”.
* Prevention
4. The most obvious thing is use common sense. If they didn’t need it at Solvay in 1927, you probably don’t need it now.
For example, avoid weird hyper-personal psychological interventions (circling, debugging, etc). Therapy is a regulated profession for good reasons, and the evidence-based therapies we know about have safeguards (e.g., asymmetric privacy, theraputic alliance, regulations about sexual activity and business relationships, boards that manage complaints, etc.)
5. Obey the law. Don’t allow underage drinking, illegal drug use, drunk driving, etc., and don’t allow others to allow it. Have a zero tolerance policy on this (if that feels like a buzzkill, you can say it has to do with liability).
The reason for this is (IMO) actually quite interesting. It’s not that the law is necessarily a good guide to morals. It’s more that abusers tend to be out of control (because they have a psychological disorder, because they think they’re above or beyond ordinary requirements, or because they’re abusing drugs themselves, etc), and violating the law is a sign of this.
The extreme example I know of is the high school personality cult I mentioned above. The colleague in question was (in retrospect) terrifying: he engaged in animal abuse and setting fires (two of the Macdonald triad), and the young men in his cult engaged in sexual abuse.
In the end, however, he was “busted” (fired) for statutory rape. The other stuff going on was too fuzzy, gradual, and excusable to hit people’s radar at first (think boiling frog). But SR is a bright line, and if someone’s crazy enough to cross that line, it’s a signal that other things are off as well.
6. Preserve personal-professional boundaries with students/mentees. A baseline assumption is that you shouldn’t really know much about anyone’s personal life—who they’re dating, what their mental problems are, what kind of sex they like. It’s not forbidden knowledge, but if you (or someone else in the org) does, you might ask: to what end? Is this helping them thrive?
Similarly, respect when someone wants those boundaries, or when they want to re-establish them.
Dating and sexual relationships across the student-teacher boundary should be completely out.
* Mitigation
7. When powerful people in your group say that an abusive person is an advsior, that sends a message to vulnerable people that they ought to, or need to, tolerate abuse by that person in other contexts. If you believe a person is abusing vulnerable people to whom you, or the org, owe a duty of care, you ought to cut communication with that person.
8. Don’t give charismatic or “high performing” people a pass. There’s no real correlation between excellence and being abusive—if anything, the positive correlation with drug and mental problems makes it a negative correlation. Meanwhile, the same thing that can enable abuse (dark triad traits) can also appear as high performance.
9. Done right, none of this requires drama. Among other things, if your org is aware of (1) through (6), abusers will go elsewhere. Having zero tolerance also makes it a lot easier to help good-faith people not abuse unintentionally—you can step in before things go off the rails, when the stakes are low, and save important relationships.
EDIT: since you asked where academia is falling short. I’d say it falls short in (1) and (2), is sort of OK in (3) in part because of Title IX and similar things, is good in (4) in part because there are long-standing traditions of common sense, and in (5) because lawyers, and falls short in (6), (7), and (8).
What’s being described here seems to be violating all eight rules at different levels. Most obvious to me from the outside is (1), (4), (5), (6) and (7).
EDIT2: since this came up. Good practice is that the vulnerable person can’t waive these concerns.
For example, the answer to “but I want to do [intense weird psychological thing] with my mentor” should be “not as long as you or this mentor remains with the org”, or at the very least “not as long as this mentor remains with the org with a duty of care towards you”.
Somebody in the comments said that many of the people reporting abuse are trans, and “trans people suffer from mental illness more”, so maybe they’re just crazy and everything was actually pretty OK.
Hopefully this reasoning looks as crazy to you as it does to me; in the 1970s people would have said the same about gay people, but now we realize that a lot of that was due to homophobia (etc), and a lot of it was due to the fact that gay people, being marginalized, made soft targets for manipulation, blackmail, etc.
So, I think this is not a fair reading of the comment in question. Not a million miles away from, but far enough that I wanted to point it out.
But also, you seem to be saying something like: “consider that maybe trans people’s rates of mental illness are downstream of them being trans and society being transphobic, not that their transness is downstream of mental illness”.
And, okay, but...
Consider a hypothetical trans support forum. If rationalistthrowaway is right, you’d expect the members of that forum to have higher than average rates of mental illness, possibly leading to high profile events like psychotic breaks and suicides. And it sounds like you don’t disagree with this?
(Like, it sounds like you might want to add “some of what gets diagnosed in mental illness in trans people is just the diagnostic machinery being transphobic, and that accounts for some of the increase”. Sure, stipulated. But it doesn’t sound like you’d say that accounts for all of the increase.)
Then someone sees these high profile events, and wonders what’s going on here? Is it something about the forum that’s triggering them? And someone else points out that trans people have a high baseline rate of mental illness, and that seems highly relevant.
It seems to me that your reply would be just as fitting there. Which is to say, I think it misunderstands the point being made; and also (through social punishment) makes it less likely that people will be able to figure out what’s going on and be able to make things better.
Thanks so much for the response! I really appreciate it.
I’m assuming your institution wants to follow an academic model, including teaching, mentorship, hierarchical student-teacher relationships, etc.
I think we have more of a standard manager-managee hierarchal relationship, with the normal corporate guardrails plus a few more. We also have explicit lines of reporting for abuse or other potential issues to people outside of the organization to minimize potential coverups.
Here are my general thoughts:
An open question is when you have a duty of care
I’m kind of confused. Surely organizations by default have a power dynamic over employees, and managers over reports, and abusing this is bad? Maybe I’m confused and you mean a stronger thing by “duty of care”
Seems straightforwardly true to me, though I think you’re maybe underestimating correlates of direct harm. (eg I expect in many of the cases cited, there’s things like megalomania, insufficient humility, insufficient willingness to listen to contrary evidence, caring more about charismatic personalities than object-level arguments, etc)
Speaking as someone in the subset of “women and minorities”, I’d be pretty concerned about any form of special treatments or affordances given because “women and minorities” are at higher risk, aside from really obvious ones like being moderately more careful about male supervisor/female supervisee.
In particular, this creates bad dynamics/incentive structures, like making it less likely to provide honest/critical feedback to “marginalized” groups, which is one of the things I was warned against in management training.
This seems correct. Also you want multiple trusted points of contact outside the organization, which I think both academia and rationality are failing at.
EA organizations often have Julia Wise, but she’s stretched too thin and thus have (arguably) made significant mistakes as a result, as pointed out in a different thread.
This seems right to me. I think “common sense” should be dereferenced a little for people coming from different cultures, but the company culture of the AngloAmerican elite seems not-crazy as a starting point.
I think it’s Very Bad to allow most forms of lawbreaking on “work time.” But I think you’re implying something much stronger than that, and (speaking as someone who think all recreational drugs are dumb and straightforwardly do not pass any cost-benefits analysis, and have consumed less than a bottle of wine in my entire life) I really don’t think it’s the job of a workplace to police employee’s time off, regardless of whether it’s doing recreational drugs or listening to pirated music.
maybe it’s different if jobs are in person?
But I once worked at a company which had in our code-of-contact that employees can’t drink in parties with other employees, and even though I had no inclination to drink, I still thought that was clearly too crazy/controlling
This seems right. Most companies have rules against managers dating subordinates, and I think for probably good reasons.
This sounds right, though “if you believe” is a probabilistic claim, and if I think the base rate is 5%, I’m not sure you think cutting communication should have at 15% (already ~3x elevated risk!) or 75% or 95%.
I think I agree? But I think your reasoning is shoddy here. “There’s no real correlation between excellence and being abusive” is a population claim, but obviously what people are evaluating is usually individuals.
“Among other things, if your org is aware of (1) through (6), abusers will go elsewhere” One thing I’m confused about is if an organization has credible Bayesian evidence (say 40% is the cutoff) that an employee abuses their reports, it may make sense for the organization to fire them, way before there’s enough evidence to convict in a court of law. But it’s unclear what you should do in the broader ecosystem.
In academia my impression is that professors often switch universities after charges of suspicion, which seems not ideal and not what I’d want to replicate.
This seems like the beginning of a very good discussion, but:
I want to be clear that I’m not a member of the LW community, and I don’t want to take up space here.
There are complex and interesting ideas in play on both sides that are hard to communicate in a back-and-forth, and are perhaps better saved for a structured long-form presentation.
To that end, I’ll suggest that if you like we chat offline. I’m in NYC, for example, and you’re welcome to get in touch via PM.
What I’m talking about is a system of moral duties and obligations connected to an explicitly academic mission. Academia is older than the corporation, and is a separate world. It’s very important not to confuse them, and I wish that corporations (and “research labs” associated with corporations) would state very clearly “we are in no way an academic institution”.
To be clear, my own organization is a nonprofit. We are not interested in making money, nor in doing other things of low moral value.
I currently think emulating the culture of normal companies is a better starting template than academia or other research nonprofits (many of whom have strong positions that they want to believe and research that oh-so-interestingly happen to justify their pre-existing beliefs), though of course different cultures have different poisons that are more or less salient to different people.
Women and people from marginalized groups are at much higher risk. They’re less able to trust a random stranger, and they’re also less able to appeal to social norms or law enforcement. They also are at higher risk if they do report.
That seems to me doubtful. Relative to viticization survey reported rape numbers women seem to be much more willing to report it if they get raped then men.
A woman who reports sexual harrassment from a male mentor has it radically easier then a man who reports sexual harrassment from a female mentor.
(this does not deminish the fact that it’s worth listening to reports from women, but the mental model behind believing that it’s easy to report for men is wrong)
Thanks for the outside perspective. If you’re willing to go into more detail, I’m interested in a more detailed account from you on both what academia’s safeguards are and (per gwillen’s comment) where do you think academia’s safeguards fall short and how that can be fixed.
This is decision-relevant to me as I work in a research organization outside of academia (though not working on AI risk specifically), and I would like us to both be more productive than typical in academia and have better safeguards against abuse.
If it helps, we have about 15 researchers now, we’re entirely remote, and we hire typically from people who just finished their PhDs or have roughly equivalent research experience, although research interns/fellows are noticeably younger (maybe right after undergrad is the median).
Sure. I’m really glad to hear. This is not my community, but you did explicitly ask.
This is just off the top of my head, and I don’t mean it to be a final complete and correct list. It’s just to give you a sense of some things I’ve encountered, and to help you and your org think about how to empower people and help them flourish. Academia uses a lot of these to avoid the geek-MOP-sociopath cycle.
I’m assuming your institution wants to follow an academic model, including teaching, mentorship, hiearchical student-teacher relationships, etc.
An open question is when you have a duty of care. My rule of thumb is (1) when you or the org is explicitly saying “I’m your teacher”, “I’m your mentor”; (2) when you feel a power imbalance with someone because this relationship has arisen implicitly; (3) when someone is soliciting this role from you, whether you want it or not.
If you’re a business making money, that’s quite different, just say “we’re going to use your body and mind to make money” and you’ve probably gotten your informed consent. :)
* Detection
1. Abuse is non-Gaussian. A small number of people may experience a great deal, while the majority see nothing wrong. That means that occasional random sampling is not going to identify problems. There are a lot of comments here saying “XYZ org (etc) was great, I saw nothing bad” — this is not a good signal.
2. Women and people from marginalized groups are at much higher risk. They’re less able to trust a random stranger, and they’re also less able to appeal to social norms or law enforcement. They also are at higher risk if they do report.
Somebody in the comments said that many of the people reporting abuse are trans, and “trans people suffer from mental illness more”, so maybe they’re just crazy and everything was actually pretty OK.
Hopefully this reasoning looks as crazy to you as it does to me; in the 1970s people would have said the same about gay people, but now we realize that a lot of that was due to homophobia (etc), and a lot of it was due to the fact that gay people, being marginalized, made soft targets for manipulation, blackmail, etc.
3. Always take reports seriously, even if “the person seems weird”.
* Prevention
4. The most obvious thing is use common sense. If they didn’t need it at Solvay in 1927, you probably don’t need it now.
For example, avoid weird hyper-personal psychological interventions (circling, debugging, etc). Therapy is a regulated profession for good reasons, and the evidence-based therapies we know about have safeguards (e.g., asymmetric privacy, theraputic alliance, regulations about sexual activity and business relationships, boards that manage complaints, etc.)
5. Obey the law. Don’t allow underage drinking, illegal drug use, drunk driving, etc., and don’t allow others to allow it. Have a zero tolerance policy on this (if that feels like a buzzkill, you can say it has to do with liability).
The reason for this is (IMO) actually quite interesting. It’s not that the law is necessarily a good guide to morals. It’s more that abusers tend to be out of control (because they have a psychological disorder, because they think they’re above or beyond ordinary requirements, or because they’re abusing drugs themselves, etc), and violating the law is a sign of this.
The extreme example I know of is the high school personality cult I mentioned above. The colleague in question was (in retrospect) terrifying: he engaged in animal abuse and setting fires (two of the Macdonald triad), and the young men in his cult engaged in sexual abuse.
In the end, however, he was “busted” (fired) for statutory rape. The other stuff going on was too fuzzy, gradual, and excusable to hit people’s radar at first (think boiling frog). But SR is a bright line, and if someone’s crazy enough to cross that line, it’s a signal that other things are off as well.
6. Preserve personal-professional boundaries with students/mentees. A baseline assumption is that you shouldn’t really know much about anyone’s personal life—who they’re dating, what their mental problems are, what kind of sex they like. It’s not forbidden knowledge, but if you (or someone else in the org) does, you might ask: to what end? Is this helping them thrive?
Similarly, respect when someone wants those boundaries, or when they want to re-establish them.
Dating and sexual relationships across the student-teacher boundary should be completely out.
* Mitigation
7. When powerful people in your group say that an abusive person is an advsior, that sends a message to vulnerable people that they ought to, or need to, tolerate abuse by that person in other contexts. If you believe a person is abusing vulnerable people to whom you, or the org, owe a duty of care, you ought to cut communication with that person.
8. Don’t give charismatic or “high performing” people a pass. There’s no real correlation between excellence and being abusive—if anything, the positive correlation with drug and mental problems makes it a negative correlation. Meanwhile, the same thing that can enable abuse (dark triad traits) can also appear as high performance.
9. Done right, none of this requires drama. Among other things, if your org is aware of (1) through (6), abusers will go elsewhere. Having zero tolerance also makes it a lot easier to help good-faith people not abuse unintentionally—you can step in before things go off the rails, when the stakes are low, and save important relationships.
EDIT: since you asked where academia is falling short. I’d say it falls short in (1) and (2), is sort of OK in (3) in part because of Title IX and similar things, is good in (4) in part because there are long-standing traditions of common sense, and in (5) because lawyers, and falls short in (6), (7), and (8).
What’s being described here seems to be violating all eight rules at different levels. Most obvious to me from the outside is (1), (4), (5), (6) and (7).
EDIT2: since this came up. Good practice is that the vulnerable person can’t waive these concerns.
For example, the answer to “but I want to do [intense weird psychological thing] with my mentor” should be “not as long as you or this mentor remains with the org”, or at the very least “not as long as this mentor remains with the org with a duty of care towards you”.
So, I think this is not a fair reading of the comment in question. Not a million miles away from, but far enough that I wanted to point it out.
But also, you seem to be saying something like: “consider that maybe trans people’s rates of mental illness are downstream of them being trans and society being transphobic, not that their transness is downstream of mental illness”.
And, okay, but...
Consider a hypothetical trans support forum. If rationalistthrowaway is right, you’d expect the members of that forum to have higher than average rates of mental illness, possibly leading to high profile events like psychotic breaks and suicides. And it sounds like you don’t disagree with this?
(Like, it sounds like you might want to add “some of what gets diagnosed in mental illness in trans people is just the diagnostic machinery being transphobic, and that accounts for some of the increase”. Sure, stipulated. But it doesn’t sound like you’d say that accounts for all of the increase.)
Then someone sees these high profile events, and wonders what’s going on here? Is it something about the forum that’s triggering them? And someone else points out that trans people have a high baseline rate of mental illness, and that seems highly relevant.
It seems to me that your reply would be just as fitting there. Which is to say, I think it misunderstands the point being made; and also (through social punishment) makes it less likely that people will be able to figure out what’s going on and be able to make things better.
Thanks so much for the response! I really appreciate it.
I think we have more of a standard manager-managee hierarchal relationship, with the normal corporate guardrails plus a few more. We also have explicit lines of reporting for abuse or other potential issues to people outside of the organization to minimize potential coverups.
Here are my general thoughts:
I’m kind of confused. Surely organizations by default have a power dynamic over employees, and managers over reports, and abusing this is bad? Maybe I’m confused and you mean a stronger thing by “duty of care”
Seems straightforwardly true to me, though I think you’re maybe underestimating correlates of direct harm. (eg I expect in many of the cases cited, there’s things like megalomania, insufficient humility, insufficient willingness to listen to contrary evidence, caring more about charismatic personalities than object-level arguments, etc)
Speaking as someone in the subset of “women and minorities”, I’d be pretty concerned about any form of special treatments or affordances given because “women and minorities” are at higher risk, aside from really obvious ones like being moderately more careful about male supervisor/female supervisee.
In particular, this creates bad dynamics/incentive structures, like making it less likely to provide honest/critical feedback to “marginalized” groups, which is one of the things I was warned against in management training.
This seems correct. Also you want multiple trusted points of contact outside the organization, which I think both academia and rationality are failing at.
EA organizations often have Julia Wise, but she’s stretched too thin and thus have (arguably) made significant mistakes as a result, as pointed out in a different thread.
This seems right to me. I think “common sense” should be dereferenced a little for people coming from different cultures, but the company culture of the AngloAmerican elite seems not-crazy as a starting point.
I think it’s Very Bad to allow most forms of lawbreaking on “work time.” But I think you’re implying something much stronger than that, and (speaking as someone who think all recreational drugs are dumb and straightforwardly do not pass any cost-benefits analysis, and have consumed less than a bottle of wine in my entire life) I really don’t think it’s the job of a workplace to police employee’s time off, regardless of whether it’s doing recreational drugs or listening to pirated music.
maybe it’s different if jobs are in person?
But I once worked at a company which had in our code-of-contact that employees can’t drink in parties with other employees, and even though I had no inclination to drink, I still thought that was clearly too crazy/controlling
This seems right. Most companies have rules against managers dating subordinates, and I think for probably good reasons.
This sounds right, though “if you believe” is a probabilistic claim, and if I think the base rate is 5%, I’m not sure you think cutting communication should have at 15% (already ~3x elevated risk!) or 75% or 95%.
I think I agree? But I think your reasoning is shoddy here. “There’s no real correlation between excellence and being abusive” is a population claim, but obviously what people are evaluating is usually individuals.
“Among other things, if your org is aware of (1) through (6), abusers will go elsewhere” One thing I’m confused about is if an organization has credible Bayesian evidence (say 40% is the cutoff) that an employee abuses their reports, it may make sense for the organization to fire them, way before there’s enough evidence to convict in a court of law. But it’s unclear what you should do in the broader ecosystem.
In academia my impression is that professors often switch universities after charges of suspicion, which seems not ideal and not what I’d want to replicate.
This seems like the beginning of a very good discussion, but:
I want to be clear that I’m not a member of the LW community, and I don’t want to take up space here.
There are complex and interesting ideas in play on both sides that are hard to communicate in a back-and-forth, and are perhaps better saved for a structured long-form presentation.
To that end, I’ll suggest that if you like we chat offline. I’m in NYC, for example, and you’re welcome to get in touch via PM.
To be clear, my own organization is a nonprofit. We are not interested in making money, nor in doing other things of low moral value.
I currently think emulating the culture of normal companies is a better starting template than academia or other research nonprofits (many of whom have strong positions that they want to believe and research that oh-so-interestingly happen to justify their pre-existing beliefs), though of course different cultures have different poisons that are more or less salient to different people.
But yeah, let’s take this offline.
That seems to me doubtful. Relative to viticization survey reported rape numbers women seem to be much more willing to report it if they get raped then men.
A woman who reports sexual harrassment from a male mentor has it radically easier then a man who reports sexual harrassment from a female mentor.
(this does not deminish the fact that it’s worth listening to reports from women, but the mental model behind believing that it’s easy to report for men is wrong)