I’m about to start reading up on which treatments for mental disorders actually work. (Things like how CBT is significantly more effective against depression than most other therapies.) I’m also interested in things other than formal treatment that affect a person’s odds of recovery—exercise, life circumstances, etc. I expect this might be a tricky thing to research, since the variety of treatments available indicates it’s not a solved problem even within the psychology community.
I’ve never done this sort of research before, so I’m not sure how to go about it. What sorts of places should I look? (I’m a student, so I can get past paywalls—though I’m not sure whether reading academic papers is efficient.) How do I recognize a reliable source of information when I see one? What search terms should I use to find comparisons or evaluations of treatments?
Also, as a quick sanity check—I’m doing this under the assumption that most psychology and psychiatry professionals will be biased in favor of things within their own area of expertise, and that there’s a significant chance (though possibly small—I don’t know) of a medical general practitioner not knowing enough (about the field and/or the specific patient) to recommend the right kind of specialist. And so it’d be possible to learn enough by reading a lot that my knowledge could be useful in addition to that of professionals. Is this accurate? (No appointments are being put off because of this, I just want to make sure I’m not wasting my time.)
Short response: Check out the Cochrane Library on mental health. (Browse by Topics in the left-hand side, Expand, then click on Mental Health—as of just now there are 406 entries.)
Evaluating healthcare interventions is hard. The gold standard is a randomised controlled trial (RCT), published in a peer reviewed journal. But there are all sorts of problems with single trials, some of which you allude to here. It’s a really great idea to do a systematic review of all published trials and combine the good ones to get the best evidence available.
Doing this well is really hard—you need specialist expertise in the specific area to correctly interpret the primary literature (the RCTs), and specialist skills in systematic reviewing (as with RCTs themselves, there are many obvious and subtle issues about how to do them well). And it takes ages.
Luckily, there’s an international collaboration of people, called the Cochrane Collaboration who get together to do this sort of thing, and have been beavering away for 20 years.
Unless you have significant resources, you are unlikely to do better on any topic than the latest available Cochrane Review. And if you do have significant resources, you’re likely to do well to start with it.
When a health issue pops up for me or someone I care about, I jump straight for the Cochrane review (and also any relevant guidelines and protocols, but that’s a tier down the evidence quality pyramid), and it’s like I’m getting a well thought-through briefing from the world’s experts on what we currently know about what works and what doesn’t.
I love it.
As a postscript, there is a whole field of healthcare informatics that looks at how to find good academic papers on a particular issue—I once ran a whole course on the topic (and related ones). The shortcut answer is ‘use Cochrane’; the long spadework answer is ‘search Medline’.
I forgot to mention: if you have money rather than time available, there’s MetaMed, which has personnel overlaps with LW, and does the looking-up job for you for a fee. See the write-up/pitch for it by Eliezer.
He takes individual variation very seriously. Blood and saliva tests are a good idea before supplements because needs and reactions for supplements vary a lot.
He believes that the human body has evolved to be good at living, and it’s better to support it and find out what’s getting in its way rather than just trying to prevent symptoms.
At the same time, he also comes up with a scoring system for a person’s symptoms—efforts at treatment should either improve the score or be discontinued.
Possibly important for LW: He believes that very low carb diets + hard exercise with insufficient recovery + fasting all increase stress hormones, and can lead to exhaustion that takes a lot of time and work to cure, and that people don’t necessarily get a lot of warning before they’re in trouble.
Possibly important for LW: He believes that very low carb diets + hard exercise with insufficient recovery + fasting all increase stress hormones, and can lead to exhaustion that takes a lot of time and work to cure, and that people don’t necessarily get a lot of warning before they’re in trouble.
In general this seems accurate but I request clarification regarding the ‘hard exercise’ claim. The word ‘hard’ in that context sets off warning bells. While either suitably extreme instantiation of ‘hard’ or a tautological instantiation of ‘insufficient recovery’ could make this claim denotatively true the connotations are something to be wary of. In particular I would like to assert the following claims and in so doing invite disagreement if the advice from your source happens to contradict them:
High intensity exercise is better than low intensity exercise in terms of effect on stress hormones. For a given degree of physiological distress and fatigue induced better stress hormone results come from high intensity exercise than low intensity exercise.
For the prevention of overtraining—the state where recovery is insufficient to the demands of exercise—an often effective solution is to reduce the amount of low intensity exercise and replace it with high intensity exercise. Obviously there will be less exercise as measured by time but more by measures of physical adaptation.
Except in extreme cases the effect of high intensity exercise on stress hormones is decidedly positive. Not only does such exertion reduce the level of stress hormones (and stress) in the body on average it also makes the body better able to resist the effects of stress. In particular the new neurons produced due to high intensity exercise are less vulnerable to destruction via chronic or acute stress. Many rats died in the discovery of this phenomenon.
I don’t wish to criticise or reject your source. Pardon me if I am excessively wary of the wording. Unless his advice contradicts the above claims, what you have conveyed of his lessons thus far indicates that he is worth listening to. I’ll add that his observation about fasting matches my research on the subject. Those that eat irregularly can in general improve their cortisol levels by consuming protein in the morning—that seems to be a fairly consistent finding.
From memory, since I don’t want to hack through the podcast again: He came up with an extreme example (something like Crossfit + an hour of cardio a day) and then clarified by saying that Crossfit can be very good, but it’s important to have a coach rather than do it on your own, and also something to the effect while it’s better to get blood tests and such, you have a clue that you’re pushing yourself too hard if you’re developing sleep problems.
From a different source: Serious problems with “Strong is the new skinny”—take a took at the number of comments from coaches who say they have to restrain their clients from over-training. I think there’s a cultural problem.
The more I learn about human metabolism, the more I realize how complex it is. And because of that, I am more skeptical of simple, broad claims, that aren’t backed up by solid research.
I wasn’t familiar with Cochrane; that looks like an excellent resource. Unfortunately, it looks like a lot of summaries haven’t been updated in a decade—is this something to be worried about, and if so, is there another resource someone can recommend other than simply reading PubMed and doing your own meta-analysis?
On the more personal question of what to do here to get the best treatment:
You care more about your condition than any medic, and so are motivated to spend more time on it. However, a general practitioner will have much more experience than you in diagnosing and treating mental health issues in the general population, and a specialist in mental health will know more about their area specifically than you can realistically hope to. (Unless you have a very rare condition, have the background chops to be a doctor, and put in an awful lot of work—which some people do.)
My guess is that you can almost certainly learn enough to be very helpful in decision-making about your condition. I wouldn’t bother with the primary literature though—go for Cochrane Reviews and NICE Guidelines (here’s the ones for mental health), and recent textbooks.
(NICE is a UK health body that basically takes the sort of information you get from Cochrane Reviews about what works and does some clever sums to work out what makes sense to do in yielding the highest number of QALYs given a fixed budget for healthcare.)
It’s not always an easy ride—some clinicians have a morbid fear of patients who have ‘consulted Dr Google’ - and not without good reason.
But if you go along seeing yourself as an informed patient trying to engage in shared decision-making, most are happy about it in my experience. (Though my experience is limited to the UK.) If they’re threatened by you mentioning this sort of evidence, I’d take it as a clear sign to change (if you possibly can).
It also puts you right on the sharp end of the distinction between what’s true of a population and what’s true for you.
My guess is that you can almost certainly learn enough to be very helpful in decision-making about your condition.
“Helpful in decision-making” is a curious way of speaking about mental health. In it’s core mental health is about changing what the patient does and not about the doctor.
For a lot of mental health issues it matters whether a patient feels agency.
My guess is that you can almost certainly learn enough to be very helpful in decision-making about your condition. I wouldn’t bother with the primary literature though
Don’t underrate primary literature. Reading it can help you to build understanding of what the disease is about. There are a lot of details that get stripped out in reviews.
“Helpful in decision-making” is a curious way of speaking about mental health.
For clarity, I meant “learn enough to be very helpful to you in your decision-making about your condition”, rather than useful to the doctor. (Which is not to say that it might not also be helpful to the doctor.)
For a lot of mental health issues it matters whether a patient feels agency.
Yes, absolutely—so for mental health issues it can be particularly helpful to learn enough about your situation and possible interventions so that you can be more involved in the decisions about them, rather than the locus of control lying with the clinician.
If you haven’t picked it up already, Feeling Good is a very good introduction to CBT. Also, according to its introduction, studies have shown that reading the book and doing the homework is surprisingly effective at treating depression on its own, in many cases. But maybe you already are up to speed on that stuff.
What’s you goal? Do you want to help a specific person or do you want to gather general knowledge.
A clinical trial can tell you whether a treatment works for the average person. It however can’t tell you whether it works for a specific person.
When it comes to helping a specific person with a serious issue the correct solution is often about trying multiple approaches. If you try 9 things and one works you win.
When it comes to therapist, it’s important to have good chemistry with the therapist. If you don’t fell comfortable with a certain therapist they probably won’t be able to help you even if the practice a technique with has studies to back it up.
Thank you for the advice—these all seem like good ideas. Multiple approaches is what I was thinking—I was going to find out what’s out there, and start with the ones that seem the most likely to work and don’t conflict with each other or add up to be too costly (for example, you can only do so many time-consuming things concurrently).
I have a specific person in mind, and am intentionally being vague about whether it’s me or someone else and who. I know it’s not very polite to ask for help with a problem without saying what the problem is, and I’m sorry for doing that. I was iffy about posting at all for privacy reasons, but I knew that good advice on this would be really valuable, so the best I could come up with was posting but giving minimal information.
I was going to find out what’s out there, and start with the ones that seem the most likely to work and don’t conflict with each other or add up to be too costly (for example, you can only do so many time-consuming things concurrently).
In this case I think it makes sense to try some simple methods even if they don’t have peer reviewed research behind them.
My personal readings on.. pretty much this exact subject .. is that there are a handful of effective drugs for specific—easily identifiable—problems and a larger number of drugs that mostly just calm you the heck down, and that misdiagnosis is a really major problem for both effective treatment and the development of treatments.
If you are bi-polar, then any doctor or shrink worth his salt is going to diagnose that correctly, and lithium will help. If you have a problem with a less clear cut set of symptoms?
You are going to wind up with a diagnosis. Some diagnosis. Which may, or may not, be even remotely right, but now it is in your medical file, and all further symptoms are read through that filter. It is very rare that a diagnosis gets revised. This in turn makes developing treatments for mental issues a lot harder. When you are trying to develop a cure for a cancer or an infection, the group you test your drugs on will not in fact instead be suffering from fracking lupus. If you are testing a drug or other treatment for any given mental problem? Yhea, if you are really lucky, then half your test group has the actual problem you wish to treat.
You’re probably somewhat overoptimistic about how good medical diagnosis is (look at the problems around Lyme disease), but it’s plausible that the situation is much worse for psychological problems.
This is a really, really good point that I hadn’t thought of. I remember hearing in an intro to psychology class that psychologists have a worse misdiagnosis rate (or rather, rate of diagnosis being changed later—not necessarily to the correct thing even then) than emergency rooms, but I hadn’t thought of how that would affect research.
I’m about to start reading up on which treatments for mental disorders actually work. (Things like how CBT is significantly more effective against depression than most other therapies.) I’m also interested in things other than formal treatment that affect a person’s odds of recovery—exercise, life circumstances, etc. I expect this might be a tricky thing to research, since the variety of treatments available indicates it’s not a solved problem even within the psychology community.
I’ve never done this sort of research before, so I’m not sure how to go about it. What sorts of places should I look? (I’m a student, so I can get past paywalls—though I’m not sure whether reading academic papers is efficient.) How do I recognize a reliable source of information when I see one? What search terms should I use to find comparisons or evaluations of treatments?
Also, as a quick sanity check—I’m doing this under the assumption that most psychology and psychiatry professionals will be biased in favor of things within their own area of expertise, and that there’s a significant chance (though possibly small—I don’t know) of a medical general practitioner not knowing enough (about the field and/or the specific patient) to recommend the right kind of specialist. And so it’d be possible to learn enough by reading a lot that my knowledge could be useful in addition to that of professionals. Is this accurate? (No appointments are being put off because of this, I just want to make sure I’m not wasting my time.)
Short response: Check out the Cochrane Library on mental health. (Browse by Topics in the left-hand side, Expand, then click on Mental Health—as of just now there are 406 entries.)
Evaluating healthcare interventions is hard. The gold standard is a randomised controlled trial (RCT), published in a peer reviewed journal. But there are all sorts of problems with single trials, some of which you allude to here. It’s a really great idea to do a systematic review of all published trials and combine the good ones to get the best evidence available.
Doing this well is really hard—you need specialist expertise in the specific area to correctly interpret the primary literature (the RCTs), and specialist skills in systematic reviewing (as with RCTs themselves, there are many obvious and subtle issues about how to do them well). And it takes ages.
Luckily, there’s an international collaboration of people, called the Cochrane Collaboration who get together to do this sort of thing, and have been beavering away for 20 years.
Unless you have significant resources, you are unlikely to do better on any topic than the latest available Cochrane Review. And if you do have significant resources, you’re likely to do well to start with it.
When a health issue pops up for me or someone I care about, I jump straight for the Cochrane review (and also any relevant guidelines and protocols, but that’s a tier down the evidence quality pyramid), and it’s like I’m getting a well thought-through briefing from the world’s experts on what we currently know about what works and what doesn’t.
I love it.
As a postscript, there is a whole field of healthcare informatics that looks at how to find good academic papers on a particular issue—I once ran a whole course on the topic (and related ones). The shortcut answer is ‘use Cochrane’; the long spadework answer is ‘search Medline’.
Good luck.
Wow, now I’m really glad I asked! This sounds like exactly what I was looking for. Thank you!
You’re welcome. Glad to help.
I forgot to mention: if you have money rather than time available, there’s MetaMed, which has personnel overlaps with LW, and does the looking-up job for you for a fee. See the write-up/pitch for it by Eliezer.
Not quite enough of it for MetaMed, but thank you for the suggestion.
This health coach seems sensible, and might be cheaper than Metamed.
Nancy, what made you think that coach was sensible?
He takes individual variation very seriously. Blood and saliva tests are a good idea before supplements because needs and reactions for supplements vary a lot.
He believes that the human body has evolved to be good at living, and it’s better to support it and find out what’s getting in its way rather than just trying to prevent symptoms.
At the same time, he also comes up with a scoring system for a person’s symptoms—efforts at treatment should either improve the score or be discontinued.
Possibly important for LW: He believes that very low carb diets + hard exercise with insufficient recovery + fasting all increase stress hormones, and can lead to exhaustion that takes a lot of time and work to cure, and that people don’t necessarily get a lot of warning before they’re in trouble.
In general this seems accurate but I request clarification regarding the ‘hard exercise’ claim. The word ‘hard’ in that context sets off warning bells. While either suitably extreme instantiation of ‘hard’ or a tautological instantiation of ‘insufficient recovery’ could make this claim denotatively true the connotations are something to be wary of. In particular I would like to assert the following claims and in so doing invite disagreement if the advice from your source happens to contradict them:
High intensity exercise is better than low intensity exercise in terms of effect on stress hormones. For a given degree of physiological distress and fatigue induced better stress hormone results come from high intensity exercise than low intensity exercise.
For the prevention of overtraining—the state where recovery is insufficient to the demands of exercise—an often effective solution is to reduce the amount of low intensity exercise and replace it with high intensity exercise. Obviously there will be less exercise as measured by time but more by measures of physical adaptation.
Except in extreme cases the effect of high intensity exercise on stress hormones is decidedly positive. Not only does such exertion reduce the level of stress hormones (and stress) in the body on average it also makes the body better able to resist the effects of stress. In particular the new neurons produced due to high intensity exercise are less vulnerable to destruction via chronic or acute stress. Many rats died in the discovery of this phenomenon.
I don’t wish to criticise or reject your source. Pardon me if I am excessively wary of the wording. Unless his advice contradicts the above claims, what you have conveyed of his lessons thus far indicates that he is worth listening to. I’ll add that his observation about fasting matches my research on the subject. Those that eat irregularly can in general improve their cortisol levels by consuming protein in the morning—that seems to be a fairly consistent finding.
From memory, since I don’t want to hack through the podcast again: He came up with an extreme example (something like Crossfit + an hour of cardio a day) and then clarified by saying that Crossfit can be very good, but it’s important to have a coach rather than do it on your own, and also something to the effect while it’s better to get blood tests and such, you have a clue that you’re pushing yourself too hard if you’re developing sleep problems.
From his blog: On combining fasting and exercise.
From a different source: Serious problems with “Strong is the new skinny”—take a took at the number of comments from coaches who say they have to restrain their clients from over-training. I think there’s a cultural problem.
Fasting can be bad for people with disregulated cortisol.
Thanks NancyLebovitz.
The more I learn about human metabolism, the more I realize how complex it is. And because of that, I am more skeptical of simple, broad claims, that aren’t backed up by solid research.
I wasn’t familiar with Cochrane; that looks like an excellent resource. Unfortunately, it looks like a lot of summaries haven’t been updated in a decade—is this something to be worried about, and if so, is there another resource someone can recommend other than simply reading PubMed and doing your own meta-analysis?
On the more personal question of what to do here to get the best treatment:
You care more about your condition than any medic, and so are motivated to spend more time on it. However, a general practitioner will have much more experience than you in diagnosing and treating mental health issues in the general population, and a specialist in mental health will know more about their area specifically than you can realistically hope to. (Unless you have a very rare condition, have the background chops to be a doctor, and put in an awful lot of work—which some people do.)
My guess is that you can almost certainly learn enough to be very helpful in decision-making about your condition. I wouldn’t bother with the primary literature though—go for Cochrane Reviews and NICE Guidelines (here’s the ones for mental health), and recent textbooks.
(NICE is a UK health body that basically takes the sort of information you get from Cochrane Reviews about what works and does some clever sums to work out what makes sense to do in yielding the highest number of QALYs given a fixed budget for healthcare.)
It’s not always an easy ride—some clinicians have a morbid fear of patients who have ‘consulted Dr Google’ - and not without good reason.
But if you go along seeing yourself as an informed patient trying to engage in shared decision-making, most are happy about it in my experience. (Though my experience is limited to the UK.) If they’re threatened by you mentioning this sort of evidence, I’d take it as a clear sign to change (if you possibly can).
It also puts you right on the sharp end of the distinction between what’s true of a population and what’s true for you.
“Helpful in decision-making” is a curious way of speaking about mental health. In it’s core mental health is about changing what the patient does and not about the doctor.
For a lot of mental health issues it matters whether a patient feels agency.
Don’t underrate primary literature. Reading it can help you to build understanding of what the disease is about. There are a lot of details that get stripped out in reviews.
For clarity, I meant “learn enough to be very helpful to you in your decision-making about your condition”, rather than useful to the doctor. (Which is not to say that it might not also be helpful to the doctor.)
Yes, absolutely—so for mental health issues it can be particularly helpful to learn enough about your situation and possible interventions so that you can be more involved in the decisions about them, rather than the locus of control lying with the clinician.
If you haven’t picked it up already, Feeling Good is a very good introduction to CBT. Also, according to its introduction, studies have shown that reading the book and doing the homework is surprisingly effective at treating depression on its own, in many cases. But maybe you already are up to speed on that stuff.
What’s you goal? Do you want to help a specific person or do you want to gather general knowledge.
A clinical trial can tell you whether a treatment works for the average person. It however can’t tell you whether it works for a specific person.
When it comes to helping a specific person with a serious issue the correct solution is often about trying multiple approaches. If you try 9 things and one works you win.
When it comes to therapist, it’s important to have good chemistry with the therapist. If you don’t fell comfortable with a certain therapist they probably won’t be able to help you even if the practice a technique with has studies to back it up.
Thank you for the advice—these all seem like good ideas. Multiple approaches is what I was thinking—I was going to find out what’s out there, and start with the ones that seem the most likely to work and don’t conflict with each other or add up to be too costly (for example, you can only do so many time-consuming things concurrently).
I have a specific person in mind, and am intentionally being vague about whether it’s me or someone else and who. I know it’s not very polite to ask for help with a problem without saying what the problem is, and I’m sorry for doing that. I was iffy about posting at all for privacy reasons, but I knew that good advice on this would be really valuable, so the best I could come up with was posting but giving minimal information.
In this case I think it makes sense to try some simple methods even if they don’t have peer reviewed research behind them.
http://curetogether.com/conditions lists for most conditions the treatments that patients found useful.
I was also thinking this—stuff like changing diet is pretty easy to try out.
My personal readings on.. pretty much this exact subject .. is that there are a handful of effective drugs for specific—easily identifiable—problems and a larger number of drugs that mostly just calm you the heck down, and that misdiagnosis is a really major problem for both effective treatment and the development of treatments.
If you are bi-polar, then any doctor or shrink worth his salt is going to diagnose that correctly, and lithium will help. If you have a problem with a less clear cut set of symptoms? You are going to wind up with a diagnosis. Some diagnosis. Which may, or may not, be even remotely right, but now it is in your medical file, and all further symptoms are read through that filter. It is very rare that a diagnosis gets revised. This in turn makes developing treatments for mental issues a lot harder. When you are trying to develop a cure for a cancer or an infection, the group you test your drugs on will not in fact instead be suffering from fracking lupus. If you are testing a drug or other treatment for any given mental problem? Yhea, if you are really lucky, then half your test group has the actual problem you wish to treat.
You’re probably somewhat overoptimistic about how good medical diagnosis is (look at the problems around Lyme disease), but it’s plausible that the situation is much worse for psychological problems.
This is a really, really good point that I hadn’t thought of. I remember hearing in an intro to psychology class that psychologists have a worse misdiagnosis rate (or rather, rate of diagnosis being changed later—not necessarily to the correct thing even then) than emergency rooms, but I hadn’t thought of how that would affect research.