Hm, looks to me like you committed the noncentral fallacy twice. “Scientific evidence”—evidence can still be evidence without being “scientific”. “Quackery”—yeah, there are people who scam others with fake medical advice with little regard to the others’ actual health. That doesn’t mean that all medical advice from non-doctors should be disregarded. For example, I resolved a long-standing repetitive strain injury thanks to this guy. If I decided to disregard his advice because he wasn’t a doctor, I’d probably still be controlling my computer using voice recognition (did this for almost a year and it seriously sucked).
For what it’s worth, I attribute the vast improvement in my health over the last year to quackery (that is to say, advice given by non-doctors, based on assumptions that are far from being the consensus among those within traditional scientific institutions like medical associations and universities).
I’m sure you could find literaly millions of personal anecdotes of people who allegedly got better because of homeopathy, chromotherapy, magnotherapy, faith healing and whatever form of snake oil out there.
The effectiveness of medical procedures is essentially impossible to evaluate subjectively due to large aleatory effects, individual differences, sponteneous regression and the placebo effect. On the other hand, due to the large emotional effects of illness and death, medicine is the ideal fertile ground for fallacies such as wishful thinking, confirmation bias and for outright fraud.
Therefore, the value of medical advice unsupported by science is virtually zero. In fact, it could be argued that the existence of an unscientific medical practice in a society where scientific medicine is available is actually weak evidence that such practice is ineffective and possibly fraudulent.
You’re discounting the possibility that “fringe” medical advice would have theoretical evidence behind it, as opposed to empirical evidence.
You’re also not doing an expected utility calculation. What are the costs and benefits of following this particular bit of “fringe” medical advice? In my case, the cost was: I spent a few hours reading and massaging my arm, and discovered a knot in my bicep such that when I massaged it, my wrist pain was replicated. Benefits? I got my career back. Indeed, when I first heard about “trigger points” I thought the probability that they were legit was extremely low. But in retrospect, I’m glad I followed up all the low-probability leads that I did. In fact, I wish I’d done this sort of experimentation more and sooner—the cost-benefit analysis favored it overwhelmingly. (Unfortunately, severe depression seems to make it very hard for me to motivate myself to do things that I know have only a low probability of working.)
In fact, it could be argued that the existence of an unscientific medical practice in a society where scientific medicine is available is actually weak evidence that such practice is ineffective and possibly fraudulent.
The fact that someone is passing it off as medical advice makes the probability of it being useful medical advice way, way higher than the probability that some random string of characters is useful medical advice. I agree that “fringe” medical advice is less likely to be useful than advice you get from doctors.
You’re discounting the possibility that “fringe” medical advice would have theoretical evidence behind it, as opposed to empirical evidence.
Are you talking about a theory rooted in solid biological and medical science or some alternative newagey theory like chakra points? Even within the realm of scientific theories, the ability to predict the actual effectiveness and safety of a therapy is generally limited: 92% of drugs that pass in vitro tests fail animal or human trials, and these are only the drugs that have already passed the computer-based design phase.
You’re also not doing an expected utility calculation. What are the costs and benefits of following this particular bit of “fringe” medical advice? In my case, the cost was: I spent a few hours reading and massaging my arm, and discovered a knot in my bicep such that when I massaged it, my wrist pain was replicated.
As far as I know, there are a number of conditions, notably inflammation, that are actually made worse by massage.
The fact that someone is passing it off as medical advice makes the probability of it being useful medical advice way, way higher than the probability that some random string of characters is useful medical advice.
That’s an irrelevant comparison, since nobody suggested to sample random strings for medical advice. In any case, an alleged medical advice also has a probability of being actually harmful way way higher than that of a random string.
Any form of depression that requires treatment, in particular major depression, is recognized as a severe medical condition that can be disabling and in some cases fatal, since depressed people have a significant suicide risk. Thus, any advice on how to treat depression is medical advice.
Moreover, you talk about experience with your clients, and if I understand correctly, you actually work as some sort of depression counselor. Your website even says “It’s like therapy, but better.” Therefore, you are indeed giving professional medical advice that is not based on scientific evidence.
Sorry if this comes across rude, but it seems to me that what you are doing meets the definition of quackery.
I’m not an expert, but as far as I know, depression is very hard to treat and the only treatments that have shown some effectiveness supported by scientific evidence are antidepressants and cognitive behavioral therapy, and AFAIK, even antidepressants show a weak effect.
The old school psych industry generally doesn’t work. I went through a decade of professional therapy with professional therapists that did nothing for me, and only had things turn around once I started getting into other forms of help such as cutting edge Buddhist related philosophies such as life coaching. I actually help people when people who go through our “legal” fucked up system generally don’t improve and often get worse. Most information out there is misleading and incomplete, including professional studies. The psych industry is a mess. The DSM is terrible, most therapists don’t even agree on diagnoses. I was misdiagnosed personally for depression and given a drug that caused me to become manic, in a way that shifted my baseline psychological state permanently and caused me very bad problems for many years, as the result of taking the advice of a very prestigious professional psychiatrist who works at Stanford.
Basically, the whole field is fucked. I’m doing and sharing what I find most effective, which I actually do get results from. You can look at the testimonials on my site if you want, it seems you found them. I’m collecting more testimonials. I know what I am doing is not perfect, but I think its among the best, and that is the best I can hope for, and I would rather help people than not do anything because I’m waiting for perfection that is never going to come while people are suffering.
If you work with people who didn’t find the standard interventions helpful, and they find your assistance helpful, that doesn’t mean your thing works better—it means it works better on a group filtered for finding standard interventions unhelpful.
My group is mixed. Some didn’t find standard interventions helpful, some found them somewhat helpful and then improved more working with me. Its actually more a filter of people who think similarly enough to me to hire me. But I have also worked with random friends of friends recommendations who improved, who I think are far less like the normal cluster that is likely to find me.
Its actually more a filter of people who think similarly enough to me to hire me.
I think this is close, and fortunately it is (roughly speaking) the same kinds people who are likely to read your post and take it on board. Others would be more likely to gloss over it because it isn’t as salient to them.
that doesn’t mean your thing works better—it means it works better on a group filtered for finding standard interventions unhelpful.
False dichotomy, it’s evidence for both. One conclusion might be false and the other true but other arguments are required for you to get to that point.
False dichotomy, it’s evidence for both. One conclusion might be false and the other true but other arguments are required for you to get to that point.
Yes, it is probably evidence for both, depending somewhat on what people’s beliefs are about how those in the subset likely differ from the others in the superset with regards to relative response to interventions.
However, I wouldn’t say Alicorn was presenting a dichotomy. Sure, I would have said “that doesn’t necessarily mean” in the first case and “but it does mean” in the second just for extra specificity but I wouldn’t say that is required.
However, I wouldn’t say Alicorn was presenting a dichotomy. Sure, I would have said “that doesn’t necessarily mean” in the first case and “but it does mean” in the second just for extra specificity but I wouldn’t say that is required.
As the article I linked points out, the actual penetration of science-based psychotherapy even among professionals is low, and certainly it was even lower one decade ago. Hence, your experience is far from surprising.
However, the proper course of action is to push for science-based psychotherapy, not to go for DIY or New Age (“cutting edge Buddhist related philosophies”) systems that have possibly even less scientific support than mainstream methods.
Sure, you have experience that your method works. Freud had experience that his psychoanalysis worked. So did Jung. And Moniz and Freeman with their lobotomy. They even got testimonials and Moniz even won a Nobel Prize. But these practices were not supported by scientific evidence, and they ultimately proved ineffective and, in the case of lobotomy, actively harmful.
Beware of sponteneous regression (1/3 of mental health patients get better even without therapy), the placebo effect and the confirmation bias (researchers, even professional scientists, tend to rememeber favorable outcomes and forget unfavorable ones). There is a good reason why medical experimental protocols are so strict and complex.
I do not know of any evidence of Freud or others using techniques that work. Do you know of any information that they published about patients having improvement with specific metrics and periods of time?
I do push for research and learning. I also am not going to sit on my ass and do nothing when I have techniques that I have witnessed for myself and measured as working. Life is not perfect. Science is great, but is really really far from getting a really good model of human psychology, and therapists are terrible at implementing, as we have repeatedly agreed. Doing the best with what what I can, is the most honorable thing I know how to do. I care much more about creating maximal utility in the world than being right and not making mistakes. The implication of your suggested strategy is that we should all sit around and twiddle our thumbs until perfection arrives, perhaps aside from some people in labs, who we are going to put all of our faith in humanity in.
[...]
Reread and noted you answered that they don’t have metrics. It is important to note that as I have said in other places, I am taking my own metrics. So I have evidence for myself that what I am doing is working. I may participate in creating studies and publishing eventually, I would definitely like to see it happen, but leading a study is not what I currently consider lowest hanging fruit for my contributions to humanity and taking care of myself at this point in time. Also, there are studies that are being done on Internal Family Systems and other things of that nature currently.
Yeah, and I sunk into depression despite over a decade of various lifehacking philosophical gizmos, and you don’t hear me saying that all of it is fucked and we should all just get on the happy pills. (You do hear me gushing overmuch about said happy pills.) If we’re trading anecdotes all we’re going to learn is that lots of stuff sometimes work but everything usually fails.
Obviously you’re helping people, so go you, and obviously you’re helping people more than conventional therapy because conventional therapy is just paying someone to kvetch at them regularly and that can only possibly help if you have no friends to do that with. (The advantage is that you can be suicidal and it’s their problem because it’s their job, whereas if a friend notices you’re suicidal it’s a suicide threat and you are a bad evil manipulative person and I should stop reading Captain Awkward.)
But I see no evidence that your coaching should be a first resort for mild depression, rather than a n-th resort after conventional means have failed or proven insufficient. And for major depression (where meds are much more useful than for mild depression anyway), what kind of presentation of major depression leaves you able to make big life changes?
Obviously you’re helping people, so go you, and obviously you’re helping people more than conventional therapy because conventional therapy is just paying someone to kvetch at them regularly and that can only possibly help if you have no friends to do that with.
Some of the conventional therapists are good to chat with and making it an economic transaction rather than talking about all the negative stuff with your friends can reduce the extent that you are a drain on them or the relationships. Trade is a wonderful thing!
For some reason, I found it cute that you felt the need or had the reflex to succinctly throw out the intended meaning of “straw man argument”.
I wouldn’t exactly rate high my prior that someone on LessWrong would misunderstand what you’re talking about when you use that specific term, which I suspect was the source of my reaction.
My experience with people in the community is almost entirely talking with people in person—I’ve read very little of the blog, so I’m used to more one on one conversation that is customized than the blog conventions. I was wrong that this article would be something the general LW crowd would like, so I’m now being more cautious about other assumptions about shared perspectives.
I can certainly see people’s objections about perspectives being shared without evidence. Unfortunately my personal evidence mostly comes from working with clients, which is a small sample and data I cannot share. My clients almost all improve over time, and I am taking stats, but not stats I can share, and the stats I’m taking are about general improvement and don’t isolate progress resulting from each of the specific techniques I use. I would love more data myself, and I think I’m doing a much better job of gathering data personally than any psych professional I’ve ever worked with as a client, although I’d still like to do better.
One upcoming data point that will be interesting to know is the depression levels of LWers. A question about depression, and other mental issues, has been asked to be included in the upcoming Massive LW Survey Of Doom. I actually expect that, not only will we have higher levels of Asperger’s than the base rate, but that we will also have higher levels of depression.
I have read (sorry, don’t remember where) that rational people are more likely to have depression, because the cognitive biases (specifically ones about how awesome you are) that protect most people are not as active. And that females who are also rational are extremely likely to have some depressive issues. It will be interesting to see if the LW survey supports this.
“Yvain, the Knight with the Lion is a romance by Chrétien de Troyes. It was probably written in the 1170s simultaneously with Lancelot, the Knight of the Cart, and includes several references to the narrative of that poem.
It is a story of knight-errantry, with the protagonist Yvain being exiled from the favours of his lady being required to perform a number of heroic deeds before regaining her.”
Correct me if I’m wrong, but giving medical advice that is not substantiated by scientific evidence is quackery.
Hm, looks to me like you committed the noncentral fallacy twice. “Scientific evidence”—evidence can still be evidence without being “scientific”. “Quackery”—yeah, there are people who scam others with fake medical advice with little regard to the others’ actual health. That doesn’t mean that all medical advice from non-doctors should be disregarded. For example, I resolved a long-standing repetitive strain injury thanks to this guy. If I decided to disregard his advice because he wasn’t a doctor, I’d probably still be controlling my computer using voice recognition (did this for almost a year and it seriously sucked).
For what it’s worth, I attribute the vast improvement in my health over the last year to quackery (that is to say, advice given by non-doctors, based on assumptions that are far from being the consensus among those within traditional scientific institutions like medical associations and universities).
I’m sure you could find literaly millions of personal anecdotes of people who allegedly got better because of homeopathy, chromotherapy, magnotherapy, faith healing and whatever form of snake oil out there.
The effectiveness of medical procedures is essentially impossible to evaluate subjectively due to large aleatory effects, individual differences, sponteneous regression and the placebo effect. On the other hand, due to the large emotional effects of illness and death, medicine is the ideal fertile ground for fallacies such as wishful thinking, confirmation bias and for outright fraud.
Therefore, the value of medical advice unsupported by science is virtually zero. In fact, it could be argued that the existence of an unscientific medical practice in a society where scientific medicine is available is actually weak evidence that such practice is ineffective and possibly fraudulent.
You’re discounting the possibility that “fringe” medical advice would have theoretical evidence behind it, as opposed to empirical evidence.
You’re also not doing an expected utility calculation. What are the costs and benefits of following this particular bit of “fringe” medical advice? In my case, the cost was: I spent a few hours reading and massaging my arm, and discovered a knot in my bicep such that when I massaged it, my wrist pain was replicated. Benefits? I got my career back. Indeed, when I first heard about “trigger points” I thought the probability that they were legit was extremely low. But in retrospect, I’m glad I followed up all the low-probability leads that I did. In fact, I wish I’d done this sort of experimentation more and sooner—the cost-benefit analysis favored it overwhelmingly. (Unfortunately, severe depression seems to make it very hard for me to motivate myself to do things that I know have only a low probability of working.)
The fact that someone is passing it off as medical advice makes the probability of it being useful medical advice way, way higher than the probability that some random string of characters is useful medical advice. I agree that “fringe” medical advice is less likely to be useful than advice you get from doctors.
Are you talking about a theory rooted in solid biological and medical science or some alternative newagey theory like chakra points? Even within the realm of scientific theories, the ability to predict the actual effectiveness and safety of a therapy is generally limited: 92% of drugs that pass in vitro tests fail animal or human trials, and these are only the drugs that have already passed the computer-based design phase.
As far as I know, there are a number of conditions, notably inflammation, that are actually made worse by massage.
That’s an irrelevant comparison, since nobody suggested to sample random strings for medical advice. In any case, an alleged medical advice also has a probability of being actually harmful way way higher than that of a random string.
You’re not wrong.
I am not claiming to be giving professional medical advice, so it is a straw man argument—saying something true as a way to falsely invalidate.
Any form of depression that requires treatment, in particular major depression, is recognized as a severe medical condition that can be disabling and in some cases fatal, since depressed people have a significant suicide risk. Thus, any advice on how to treat depression is medical advice.
Moreover, you talk about experience with your clients, and if I understand correctly, you actually work as some sort of depression counselor. Your website even says “It’s like therapy, but better.” Therefore, you are indeed giving professional medical advice that is not based on scientific evidence.
Sorry if this comes across rude, but it seems to me that what you are doing meets the definition of quackery.
I’m not an expert, but as far as I know, depression is very hard to treat and the only treatments that have shown some effectiveness supported by scientific evidence are antidepressants and cognitive behavioral therapy, and AFAIK, even antidepressants show a weak effect.
EDIT:
Just found this article that makes the point much better than me: http://www.thedailybeast.com/newsweek/2009/10/01/ignoring-the-evidence.html
The old school psych industry generally doesn’t work. I went through a decade of professional therapy with professional therapists that did nothing for me, and only had things turn around once I started getting into other forms of help such as cutting edge Buddhist related philosophies such as life coaching. I actually help people when people who go through our “legal” fucked up system generally don’t improve and often get worse. Most information out there is misleading and incomplete, including professional studies. The psych industry is a mess. The DSM is terrible, most therapists don’t even agree on diagnoses. I was misdiagnosed personally for depression and given a drug that caused me to become manic, in a way that shifted my baseline psychological state permanently and caused me very bad problems for many years, as the result of taking the advice of a very prestigious professional psychiatrist who works at Stanford.
Basically, the whole field is fucked. I’m doing and sharing what I find most effective, which I actually do get results from. You can look at the testimonials on my site if you want, it seems you found them. I’m collecting more testimonials. I know what I am doing is not perfect, but I think its among the best, and that is the best I can hope for, and I would rather help people than not do anything because I’m waiting for perfection that is never going to come while people are suffering.
Sorry about the ranting, rough night.
If you work with people who didn’t find the standard interventions helpful, and they find your assistance helpful, that doesn’t mean your thing works better—it means it works better on a group filtered for finding standard interventions unhelpful.
My group is mixed. Some didn’t find standard interventions helpful, some found them somewhat helpful and then improved more working with me. Its actually more a filter of people who think similarly enough to me to hire me. But I have also worked with random friends of friends recommendations who improved, who I think are far less like the normal cluster that is likely to find me.
I think this is close, and fortunately it is (roughly speaking) the same kinds people who are likely to read your post and take it on board. Others would be more likely to gloss over it because it isn’t as salient to them.
False dichotomy, it’s evidence for both. One conclusion might be false and the other true but other arguments are required for you to get to that point.
Yes, it is probably evidence for both, depending somewhat on what people’s beliefs are about how those in the subset likely differ from the others in the superset with regards to relative response to interventions.
However, I wouldn’t say Alicorn was presenting a dichotomy. Sure, I would have said “that doesn’t necessarily mean” in the first case and “but it does mean” in the second just for extra specificity but I wouldn’t say that is required.
I don’t understand how it isn’t a dichotomy.
As the article I linked points out, the actual penetration of science-based psychotherapy even among professionals is low, and certainly it was even lower one decade ago. Hence, your experience is far from surprising.
However, the proper course of action is to push for science-based psychotherapy, not to go for DIY or New Age (“cutting edge Buddhist related philosophies”) systems that have possibly even less scientific support than mainstream methods.
Sure, you have experience that your method works. Freud had experience that his psychoanalysis worked. So did Jung. And Moniz and Freeman with their lobotomy. They even got testimonials and Moniz even won a Nobel Prize. But these practices were not supported by scientific evidence, and they ultimately proved ineffective and, in the case of lobotomy, actively harmful.
Beware of sponteneous regression (1/3 of mental health patients get better even without therapy), the placebo effect and the confirmation bias (researchers, even professional scientists, tend to rememeber favorable outcomes and forget unfavorable ones). There is a good reason why medical experimental protocols are so strict and complex.
No problem, best wishes.
I do not know of any evidence of Freud or others using techniques that work. Do you know of any information that they published about patients having improvement with specific metrics and periods of time?
I do push for research and learning. I also am not going to sit on my ass and do nothing when I have techniques that I have witnessed for myself and measured as working. Life is not perfect. Science is great, but is really really far from getting a really good model of human psychology, and therapists are terrible at implementing, as we have repeatedly agreed. Doing the best with what what I can, is the most honorable thing I know how to do. I care much more about creating maximal utility in the world than being right and not making mistakes. The implication of your suggested strategy is that we should all sit around and twiddle our thumbs until perfection arrives, perhaps aside from some people in labs, who we are going to put all of our faith in humanity in.
[...]
Reread and noted you answered that they don’t have metrics. It is important to note that as I have said in other places, I am taking my own metrics. So I have evidence for myself that what I am doing is working. I may participate in creating studies and publishing eventually, I would definitely like to see it happen, but leading a study is not what I currently consider lowest hanging fruit for my contributions to humanity and taking care of myself at this point in time. Also, there are studies that are being done on Internal Family Systems and other things of that nature currently.
Yeah, and I sunk into depression despite over a decade of various lifehacking philosophical gizmos, and you don’t hear me saying that all of it is fucked and we should all just get on the happy pills. (You do hear me gushing overmuch about said happy pills.) If we’re trading anecdotes all we’re going to learn is that lots of stuff sometimes work but everything usually fails.
Obviously you’re helping people, so go you, and obviously you’re helping people more than conventional therapy because conventional therapy is just paying someone to kvetch at them regularly and that can only possibly help if you have no friends to do that with. (The advantage is that you can be suicidal and it’s their problem because it’s their job, whereas if a friend notices you’re suicidal it’s a suicide threat and you are a bad evil manipulative person and I should stop reading Captain Awkward.)
But I see no evidence that your coaching should be a first resort for mild depression, rather than a n-th resort after conventional means have failed or proven insufficient. And for major depression (where meds are much more useful than for mild depression anyway), what kind of presentation of major depression leaves you able to make big life changes?
Some of the conventional therapists are good to chat with and making it an economic transaction rather than talking about all the negative stuff with your friends can reduce the extent that you are a drain on them or the relationships. Trade is a wonderful thing!
I just had a mental “D’aww” moment.
For some reason, I found it cute that you felt the need or had the reflex to succinctly throw out the intended meaning of “straw man argument”.
I wouldn’t exactly rate high my prior that someone on LessWrong would misunderstand what you’re talking about when you use that specific term, which I suspect was the source of my reaction.
My experience with people in the community is almost entirely talking with people in person—I’ve read very little of the blog, so I’m used to more one on one conversation that is customized than the blog conventions. I was wrong that this article would be something the general LW crowd would like, so I’m now being more cautious about other assumptions about shared perspectives.
I can certainly see people’s objections about perspectives being shared without evidence. Unfortunately my personal evidence mostly comes from working with clients, which is a small sample and data I cannot share. My clients almost all improve over time, and I am taking stats, but not stats I can share, and the stats I’m taking are about general improvement and don’t isolate progress resulting from each of the specific techniques I use. I would love more data myself, and I think I’m doing a much better job of gathering data personally than any psych professional I’ve ever worked with as a client, although I’d still like to do better.
One upcoming data point that will be interesting to know is the depression levels of LWers. A question about depression, and other mental issues, has been asked to be included in the upcoming Massive LW Survey Of Doom. I actually expect that, not only will we have higher levels of Asperger’s than the base rate, but that we will also have higher levels of depression.
I have read (sorry, don’t remember where) that rational people are more likely to have depression, because the cognitive biases (specifically ones about how awesome you are) that protect most people are not as active. And that females who are also rational are extremely likely to have some depressive issues. It will be interesting to see if the LW survey supports this.
Poor Yvain...
“Yvain, the Knight with the Lion is a romance by Chrétien de Troyes. It was probably written in the 1170s simultaneously with Lancelot, the Knight of the Cart, and includes several references to the narrative of that poem.
It is a story of knight-errantry, with the protagonist Yvain being exiled from the favours of his lady being required to perform a number of heroic deeds before regaining her.”