: People in and adjacent to MIRI/CFAR manifest major mental health problems, significantly more often than the background rate.
I think this is true
My main complaint about this and the Leverage post is the lack of base-rate data. How many people develop mental health problems in a) normal companies, b) startups, c) small non-profits, d) cults/sects? So far, all I have seen are two cases. And in the startups I have worked at, I would also have been able to find mental health cases that could be tied to the company narrative. Humans being human narratives get woven. And the internet being the internet, some will get blown out of proportion. That doesn’t diminish the personal experience at all. I am updating only slightly on CFAR or MIRI. And basically not at all on “things look better from the outside than from the inside.”
In particular, I want to remind people here that something like 30-40% of grad students at top universities have either clinically diagnosed depression or anxiety (link). I think given the kind of undirected, often low-paid, work that many have been doing for the last decade, I think that’s the right reference class to draw from, and my current guess is we are roughly at that same level, or slightly below it (which is a crazy high number, and I think should give us a lot of pause).
I want to remind people here that something like 30-40% of grad students at top universities have either clinically diagnosed [emphasis mine] depression or anxiety (link)
I’m confused about how you got to this conclusion, and think it is most likely false. Neither your link, the linked study, or the linked meta-analysis in the linked study of your link says this. Instead the abstract of the linked^3 meta-analysis says:
Among 16 studies reporting the prevalence of clinically significant symptoms of depression across 23,469 Ph.D. students, the pooled estimate of the proportion of students with depression was 0.24 (95% confidence interval [CI], 0.18-0.31; I2 = 98.75%). In a meta-analysis of the nine studies reporting the prevalence of clinically significant symptoms of anxiety across 15,626 students, the estimated proportion of students with anxiety was 0.17 (95% CI, 0.12-0.23; I2 = 98.05%).
Further, the discussion section of the linked^3 study emphasizes:
While validated screening instruments tend to over-identify cases of depression (relative to structured clinical interviews) by approximately a factor of two67,68, our findings nonetheless point to a major public health problem among Ph.D. students.
So I think there is at least two things going on here:
Most people with clinically significant significant symptoms do not go get diagnosed, so “clinically significant symptoms of” depression/anxiety is a noticeably lower bar than “actually clinically diagnosed”
As implied in the quoted discussion above, if everybody were to seek diagnosis, only ~half of the rate of symptomatic people would be clinically diagnosed as having depression/anxiety.
For those keeping score, this is ~12% for depression and 8.5% for anxiety, with some error bars.
Separately, I also think:
my current guess is we are roughly at that same level, or slightly below it
is wrong. My guess is that xrisk reducers have worse mental health on average compared to grad students. (I also believe this, with lower confidence, about people working in other EA cause areas like animal welfare, global poverty, or non-xrisk longtermism, as well as serious rationalists who aren’t professionally involved in EA cause areas).
Note that the pooled prevalence is 24% (CI 18-31). But it differs a lot across studies, symptoms, and location. In the individual studies, the range is really from zero to 50% (or rather to 38% if you exclude a study with only 6 participants). I think a suitable reference class would be the University of California which has 3,190 participants and a prevalence of 38%.
Sorry, am I misunderstanding something? I think taking “clinically significant symptoms”, specific to the UC system, as a given is wrong because it did not directly address either of my two criticisms:
1. Clinically significant symptoms =/= clinically diagnosed even in worlds where there is a 1:1 relationship between clinically significant symptoms and would have been clinically diagnosed, as many people do not get diagnosed
2. Clinically significant symptoms do not have a 1:1 relationship with would have been clinically diagnosed.
Well, I agree that the actual prevalence you have in mind would be roughly half of 38% i.e. ~20%. That is still much higher than the 12% you arrived at. And either value is so high that there is little surprise some severe episodes of some people happened in a 5-year frame.
The UC Berkeley study was the one that I had cached in my mind as generating this number. I will reread it later today to make sure that it’s right, but it sure seems like the most relevant reference class, given the same physical location.
I had a look at the situation in Germany and it doesn’t look much better. 17% of students are diagnosed with at least one psychical disorder. This is based on the health records of all students insured by one of the largest public health insurers in Germany (about ten percent of the population):
I feel like the paragraph you cited just seems like the straightforward explanation of where my belief comes from?
Among 16 studies reporting the prevalence of clinically significant symptoms of depression across 23,469 Ph.D. students, the pooled estimate of the proportion of students with depression was 0.24 (95% confidence interval [CI], 0.18–0.31; I2 = 98.75%). In a meta-analysis of the nine studies reporting the prevalence of clinically significant symptoms of anxiety across 15,626 students, the estimated proportion of students with anxiety was 0.17 (95% CI, 0.12–0.23; I2 = 98.05%)
24% of people have depression, 17% have anxiety, resulting in something like 30%-40% having one or the other.
I did not remember the section about the screening instruments over-identifying cases of depression/anxiety by approximately a factor of two, which definitely cuts down my number, and I should have adjusted it in my above comment. I do think that factor of ~2 does maybe make me think that we are doing a bit worse than grad students, though I am not super sure.
Sorry, maybe this is too nitpicky, but clinically significant symptoms =/= clinically diagnosed, even in worlds where the clinically significant symptoms are severe enough to be diagnosed as such.
If you instead said in “population studies 30-40% of graduate students have anxiety or depression severe enough to be clinically diagnosed as such were they to seek diagnosis” then I think this will be a normal misreading from not jumping through enough links.
Put another way, if someone in mid-2020 told me that they had symptomatic covid and was formally diagnosed with covid, I would expect that they had worse symptoms than someone who said they had covid symptoms and later tested for covid antibodies. This is because jumping through the hoops to get a clinical diagnosis is nontrivial Bayesian evidence of severity and not just certainty, under most circumstances, and especially when testing is limited and/or gatekeeped (which is true for many parts of the world for covid in 2020, and is usually true in the US for mental health).
My main complaint about this and the Leverage post is the lack of base-rate data. How many people develop mental health problems in a) normal companies, b) startups, c) small non-profits, d) cults/sects? So far, all I have seen are two cases. And in the startups I have worked at, I would also have been able to find mental health cases that could be tied to the company narrative. Humans being human narratives get woven. And the internet being the internet, some will get blown out of proportion. That doesn’t diminish the personal experience at all. I am updating only slightly on CFAR or MIRI. And basically not at all on “things look better from the outside than from the inside.”
In particular, I want to remind people here that something like 30-40% of grad students at top universities have either clinically diagnosed depression or anxiety (link). I think given the kind of undirected, often low-paid, work that many have been doing for the last decade, I think that’s the right reference class to draw from, and my current guess is we are roughly at that same level, or slightly below it (which is a crazy high number, and I think should give us a lot of pause).
I’m confused about how you got to this conclusion, and think it is most likely false. Neither your link, the linked study, or the linked meta-analysis in the linked study of your link says this. Instead the abstract of the linked^3 meta-analysis says:
Further, the discussion section of the linked^3 study emphasizes:
So I think there is at least two things going on here:
Most people with clinically significant significant symptoms do not go get diagnosed, so “clinically significant symptoms of” depression/anxiety is a noticeably lower bar than “actually clinically diagnosed”
As implied in the quoted discussion above, if everybody were to seek diagnosis, only ~half of the rate of symptomatic people would be clinically diagnosed as having depression/anxiety.
For those keeping score, this is ~12% for depression and 8.5% for anxiety, with some error bars.
Separately, I also think:
is wrong. My guess is that xrisk reducers have worse mental health on average compared to grad students. (I also believe this, with lower confidence, about people working in other EA cause areas like animal welfare, global poverty, or non-xrisk longtermism, as well as serious rationalists who aren’t professionally involved in EA cause areas).
Note that the pooled prevalence is 24% (CI 18-31). But it differs a lot across studies, symptoms, and location. In the individual studies, the range is really from zero to 50% (or rather to 38% if you exclude a study with only 6 participants). I think a suitable reference class would be the University of California which has 3,190 participants and a prevalence of 38%.
Sorry, am I misunderstanding something? I think taking “clinically significant symptoms”, specific to the UC system, as a given is wrong because it did not directly address either of my two criticisms:
1. Clinically significant symptoms =/= clinically diagnosed even in worlds where there is a 1:1 relationship between clinically significant symptoms and would have been clinically diagnosed, as many people do not get diagnosed
2. Clinically significant symptoms do not have a 1:1 relationship with would have been clinically diagnosed.
Well, I agree that the actual prevalence you have in mind would be roughly half of 38% i.e. ~20%. That is still much higher than the 12% you arrived at. And either value is so high that there is little surprise some severe episodes of some people happened in a 5-year frame.
The UC Berkeley study was the one that I had cached in my mind as generating this number. I will reread it later today to make sure that it’s right, but it sure seems like the most relevant reference class, given the same physical location.
I had a look at the situation in Germany and it doesn’t look much better. 17% of students are diagnosed with at least one psychical disorder. This is based on the health records of all students insured by one of the largest public health insurers in Germany (about ten percent of the population):
https://www.barmer.de/blob/144368/08f7b513fdb6f06703c6e9765ee9375f/data/dl-barmer-arztreport-2018.pdf
I feel like the paragraph you cited just seems like the straightforward explanation of where my belief comes from?
24% of people have depression, 17% have anxiety, resulting in something like 30%-40% having one or the other.
I did not remember the section about the screening instruments over-identifying cases of depression/anxiety by approximately a factor of two, which definitely cuts down my number, and I should have adjusted it in my above comment. I do think that factor of ~2 does maybe make me think that we are doing a bit worse than grad students, though I am not super sure.
Sorry, maybe this is too nitpicky, but clinically significant symptoms =/= clinically diagnosed, even in worlds where the clinically significant symptoms are severe enough to be diagnosed as such.
If you instead said in “population studies 30-40% of graduate students have anxiety or depression severe enough to be clinically diagnosed as such were they to seek diagnosis” then I think this will be a normal misreading from not jumping through enough links.
Put another way, if someone in mid-2020 told me that they had symptomatic covid and was formally diagnosed with covid, I would expect that they had worse symptoms than someone who said they had covid symptoms and later tested for covid antibodies. This is because jumping through the hoops to get a clinical diagnosis is nontrivial Bayesian evidence of severity and not just certainty, under most circumstances, and especially when testing is limited and/or gatekeeped (which is true for many parts of the world for covid in 2020, and is usually true in the US for mental health).
Ah, sorry, yes. Me being unclear on that was also bad. The phrasing you give is the one I intended to convey, though I sure didn’t do it.
Thanks, appreciate the update!