LessWrong is a non-evidence-based method of teaching rationality. We don’t have good evidence that someone will get more rational after reading the sequences.
You can make a reasonable theoretic argument that people will get more rational. You don’t have the kind of evidence that you need for a EBM-treatment.
In most domains where we make choices in our lives you don’t follow pratices that are supported by evidence from peer-reviewed trials.
You don’t get a haircut from a barber who practices evidence-based barbering. Even the people who pay a lot of money for their haircuts don’t. Reading scientific papers just isn’t the only way to gather useful knowledge.
Evidence-based medicine de-emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision making and stresses the examination of evidence from clinical research.
I wouldn’t want someone to practice open-heart surgery on me based on his intuition but I don’t see a problem with taking a massage from someone who read no scientific papers but who let’s themselves be guided by his intuition and who has a positive track record with other patients.
Sure, but the idea that one should explicitly go about trying to teach rationality because there are these things called biases is much younger than the idea of medicine. Doctors have had a much longer time than LessWrong to get their act together.
Doctors have had a much longer time than LessWrong to get their act together.
The idea of teaching people to think better isn’t new. Aristoteles also tried to teach a form of rationality. But even if the idea would be radically new, why would that matter?
Why should newer ideas be subject to a lower standard of evidence? Fairness? If you want to know the truth fairness has no place.
Let’s look at another example: Romantic courtship. Do you practice evidence-based courtship, when you seek a fulfilling relationship with a woman? Would you say there no other form of courtship besides evidence-based courtship?
Most humans don’t practice evidence-based courtship. Sometimes courtship doesn’t work out. You can blame it on couple not being familiar with the scientific papers that are published on the subject of human courtship.
Nobody has shown that given the couple those scientific papers improves their relationship changes. Nobody has shown with EBM like evidence standards that doctors who are in touch with the scientific literature archieve better health outcomes for their patients.
That doesn’t mean to me that EBM has no place, but I don’t see a reason to reject any approach to increase my health that isn’t backed by EBM.
There’s some scientific evidence that suggests that vitamin D is good. In the blogosphere there are people who found that taking vitamin D supplements first thing in the morning is better than taking them in the evening. There’s no trial for the timing of vitamin D supplements. I still take them first thing in the morning.
You don’t get a haircut from a barber who practices evidence-based barbering.
That’s not exactly 100.00% true—I once overheard a barber priding himself with the fact that someone once got laid the night after getting a haircut from him.
Jokes aside, barbering is evidence-based—given that it works at all, then barbers either have knowledge of how to do that hard-coded in their DNA (unlikely) or have learned to do that—using evidence (even though not in a systematized way). You can immediately see that if you use this cutting technique then your client’s hair will look this way. OTOH, a practitioner of non-evidence-based medicine cannot immediately see that giving a patient this substance diluted in 10^20 times as much water or sticking a needle in this particular spot or whatever will help cure the patient. (Likewise, musicians are normally evidence-based musicians to some extent, but astrologists are not evidence-based astrologists; can you find more examples?)
If you interpret evidence-based in the widest sense possible, the phrase sort of loses its meaning. Note that the very post you quote explains the intended contrast between systematic and statistical use of evidence versus intuition and traditional experience based human learning.
Besides, would you not say that astrologers figure out both how to be optimally vague, avoiding being wrong while exciting their readers, much the same way musicians figure out what sounds good?
If you interpret evidence-based in the widest sense possible, the phrase sort of loses its meaning. Note that the very post you quote explains the intended contrast between systematic and statistical use of evidence versus intuition and traditional experience based human learning.
Yes, but “intuition and traditional experience based human learning” is probably much less reliable in medicine than it is in barbering, so the latter isn’t a good example in a discussion about the former.
Besides, would you not say that astrologers figure out both how to be optimally vague, avoiding being wrong while exciting their readers, much the same way musicians figure out what sounds good.
:-)
Something similar could be said about practitioners of alternative medicine, though.
Yes, but “intuition and traditional experience based human learning” is probably much less reliable in medicine than it is in barbering, so the latter isn’t a good example in a discussion about the former.
The goal of barbering is to create haircuts that increase the attractiveness of the client to people besides the barber and the client. A barber might think: “All my clients look really great”, when in reality his haircuts reduce the attractiveness of the clients.
Surely, judging someone’s attractiveness using your System 1 alone is less hard than judging someone’s health using your System 1 alone, for most people in most situations?
A professional barber is likely to notice a lot of things about a haircut that the average person doesn’t see. It could be that he creates haircuts that look impressive to other barbers but don’t look good to the average person of the opposing sex who isn’t a barber.
I do think that you can get a decent assessment of someone’s backpain by asking them whether it has gotten better. Actually that’s even how most scientific studies who measure pain do it. They let the person rate their pain subjectively and when the subjective rating gets better through the drug they see it as a win.
For a lot of serious health issues it’s easy to see when a person gets better.
Most homeopathists spend more time interviewing their patients and getting a good understanding of their condition than the average mainstream doctor who takes 5 minutes per patient.
I think the barbering example is excellent—it illustrates that, while controlled experiments more or less is physics, and while physics is great, it is probably not going to bring a paradigm shift to barbering any time soon. One should not expect all domains to be equally well suited to a cut and dried scientific approach.
Where medicine lies on this continuum of suitedness is an open question—it is probably even a misleading question, with medicine being a collection of vastly different problems. However, it is not at all obvious that simply turning up the scientificness dial is going to make things better. It is for instance conceivable that there are already people treating medicine as a hard science, and that the current balance of intuition and evidence in medicine reflects how effective these two approaches are.
I am not trying to argue whether astrology is evidence-based or not. I am saying that the very inclusive definition of evidence-based which encompasses barbering is, (a) nearly useless because it includes every possible way of doing medicine and (b) probably not the one intended by the others using the term.
“Other kinds” meant “whatever mainstream medicine does that doesn’t fall under the evidence-based label,” not alternative medicine. I should’ve been clearer.
A lot of people use the term evidence-based medicine interchangeable with mainstream medicine. What’s in your opinion medicine that counts as mainstream medicine but that doesn’t count as evidence-based medicine?
That doesn’t agree with my experience. Evidence-based medicine refers to a specific and recent movement within mainstream medicine, which is much older.
If a doctor would today practice medicine the exact way it was practiced in 1950 I don’t think you would say that the doctor practices mainstream medicine.
If you define “mainstream” by the amount of people who use it than homeopathy is probably “mainstream medicine”. Even if you go by the status of the people, when the Queen uses homeopathy it’s no low status treatment.
There are two reasons why homeopathy gets classified as “alternative medicine”.
(1) It’s uses a ideological framework that goes against the reductionist world view.
(2) There’s are not enough high quality double blind studies to allow an institution such as cochrane to recommend homeopathy as a treatment.
The term Evidence-Based medicine got made up by a bunch of university professors in 1992 to describe the style of medicine that they were teaching.
At the beginning the term intentionally downplayed clinicial experience. Today most medicial schools say that they teach Evidence-Based medicine but they weakened the definition in a way that allows clinicians using their clinical experience but that still focuses on peer reviewed trials.
If you don’t pracitice medicine the way the university teach it in their normal programs than you are practicing in my opinion “alternative medicine”.
There are even dozens of scientific studies that support homeopathy. According to a report titled “Effectiveness, Safety and Cost-Effectiveness of Homeopathy in General Practice – Summarized Health Technology Assessment” commissioned by the Swiss government:
Many high-quality investigations of pre-clinical basic research proved homeopathic high-potencies inducing regulative and specific changes in cells or living organisms. 20 of 22 systematic reviews detected at least a trend in favor of homeopathy. In our estimation 5 studies yielded results indicating clear evidence for homeopathic therapy.
There are plenty people out there who can explain you why all those homeopathy studies are flawed, but on the other hand how many double blind controlled trials do you know that show that barbers can create haircuts that increase someone’s chances with the opposing sex?
But in general people do buy homeopathic medicine not because they read the report of the Swiss government and belief it. They buy it based on anecdotal evidence. They hear that some friend had success with homeopathy and then the go out and buy it.
The fact that you are ideologically opposed to homeopathy and crystal healing working, doesn’t mean that it fails to produce anecdotal evidence.
OTOH, a practitioner of non-evidence-based medicine cannot immediately see that giving a patient this substance diluted in 10^20 times as much water or sticking a needle in this particular spot or whatever will help cure the patient.
That’s wrong. If a acupuncturist puts needles in 10 people and 5 of them lose their back pain than he has “unsystematic clinical experience” that provides evidence for his treatment.
The core of evidence-based medicine is the belief that you shouldn’t use that kind of evidence for clinical decision making but that doctors should read medicial journals that report clinical trials that show whether or not a treatment works.
Likewise, musicians are normally evidence-based musicians to some extent, but astrologists are not evidence-based astrologists; can you find more examples?
Actually musicians and astrologists are very similar. Both make money with providing entertaining performances for their clients. Members of those professions who ignore evidence about what entertains their clients go out of business.
If a acupuncturist puts needles in 10 people and 5 of them lose their back pain than he has “unsystematic clinical experience” that provides evidence for his treatment.
Maybe some of those 5 would have lost their pain even without needles. Whereas the barber knows what his client would have looked like without the hair cut.
Maybe some of those 5 would have lost their pain even without needles
Right, that’s why it’s unsystematic.
In the Bayesian sense of the word, “I stuck a needle in this person and the amount of pain he reported went down” would have to be considered to be evidence that would increase the Bayesian possibility that your hypothesis that acupuncture helps back pain is correct. However, it’s not systematic, scientific evidence. To get that kind of evidence, you would have to do systematic studies of a large number of people, give some of them acupuncture and give some of them asprin, and see what the statistical result is.
I think that’s what bogging this discussion down here, is that the word “evidence” is being used in two different ways. If we were perfectly rational beings, we would be able to use either kind of evidence, but the problem is that the first kind of evidence (individual unsystematic personal experiences) tends to be warped by all kinds of biases (selection bias, especially) making it hard to use in any kind of reliable way. You use it if it’s all you have, but systematic evidence is very much preferable.
Actually musicians and astrologists are very similar. Both make money with providing entertaining performances for their clients. Members of those professions who ignore evidence about what entertains their clients go out of business.
OK, if you consider the point of astrology to be “making money”, as opposed to “predicting people’s personalities and future events”, then it is evidence-based—but then again, if you consider the point of alternative medicine to be “making money”, as opposed to “improving people’s health”, then it is evidence-based as well. (But now that Qiaochu_Yuan has made clear that it’s not alternative medicine that he was talking about, this is kind of moot, so I’ll tap out now.)
OK, if you consider the point of astrology to be “making money”, as opposed to “predicting people’s personalities and future events”
I didn’t. I advocated another goal, entertainment. I don’t know that much about astrology but I think a fair percentage of the people who do pay a astrologists do it for entertainement purposes.
Letting someone stick needles inside you, when you go to a acupuncturist is less about getting entertainement.
The kind of people who like astrology often also like other personality tests that they find in magazines. People enjoy going through those tests.
If an astrologer would tell people something about their personality that’s accurate but that those people aren’t willing to accept, I doubt he would stay long in business.
A bit like the musician who only plays music that he himself considers to be good, but that’s “too advanced” for his audience. If the musician only sees his own opinion of his work he’s not different than an astrologer who only sees whether his horoscope is good. If you call that musician “evidence-based” than the astrologer who goes after his own judgement of his work is also “evidence-based”.
But now that Qiaochu_Yuan has made clear that it’s not alternative medicine that he was talking about, this is kind of moot, so I’ll tap out now.
Why does that matter to the question whether barbers can be meaningfully to be said to practice evidence-based barbering?
Why does that matter to the question whether barbers can be meaningfully to be said to practice evidence-based barbering?
I was claiming that barbering is more evidence-based than alternative medicine, but if alternative medicine is not what’s being discussed, then even if I turned out to be right it still wouldn’t be relevant.
LessWrong is a non-evidence-based method of teaching rationality. We don’t have good evidence that someone will get more rational after reading the sequences.
You can make a reasonable theoretic argument that people will get more rational. You don’t have the kind of evidence that you need for a EBM-treatment. In most domains where we make choices in our lives you don’t follow pratices that are supported by evidence from peer-reviewed trials.
You don’t get a haircut from a barber who practices evidence-based barbering. Even the people who pay a lot of money for their haircuts don’t. Reading scientific papers just isn’t the only way to gather useful knowledge.
The term evidence-based medicine comes from a http://jama.jamanetwork.com/article.aspx?articleid=400956 published in 1992.
It says:
I wouldn’t want someone to practice open-heart surgery on me based on his intuition but I don’t see a problem with taking a massage from someone who read no scientific papers but who let’s themselves be guided by his intuition and who has a positive track record with other patients.
Sure, but the idea that one should explicitly go about trying to teach rationality because there are these things called biases is much younger than the idea of medicine. Doctors have had a much longer time than LessWrong to get their act together.
The idea of teaching people to think better isn’t new. Aristoteles also tried to teach a form of rationality. But even if the idea would be radically new, why would that matter?
Why should newer ideas be subject to a lower standard of evidence? Fairness? If you want to know the truth fairness has no place.
Let’s look at another example: Romantic courtship. Do you practice evidence-based courtship, when you seek a fulfilling relationship with a woman? Would you say there no other form of courtship besides evidence-based courtship?
Most humans don’t practice evidence-based courtship. Sometimes courtship doesn’t work out. You can blame it on couple not being familiar with the scientific papers that are published on the subject of human courtship.
Nobody has shown that given the couple those scientific papers improves their relationship changes. Nobody has shown with EBM like evidence standards that doctors who are in touch with the scientific literature archieve better health outcomes for their patients.
That doesn’t mean to me that EBM has no place, but I don’t see a reason to reject any approach to increase my health that isn’t backed by EBM.
There’s some scientific evidence that suggests that vitamin D is good. In the blogosphere there are people who found that taking vitamin D supplements first thing in the morning is better than taking them in the evening. There’s no trial for the timing of vitamin D supplements. I still take them first thing in the morning.
I think there’s some sort of rule against discussing PUA here.
Not so much a rule against it as an understanding that it consistently leads to low-quality discussion.
(Which ChristianKI didn’t do. His kind of general observation isn’t the kind that brings on the notorious failure mode of courtship moralizing.)
That’s not exactly 100.00% true—I once overheard a barber priding himself with the fact that someone once got laid the night after getting a haircut from him.
Jokes aside, barbering is evidence-based—given that it works at all, then barbers either have knowledge of how to do that hard-coded in their DNA (unlikely) or have learned to do that—using evidence (even though not in a systematized way). You can immediately see that if you use this cutting technique then your client’s hair will look this way. OTOH, a practitioner of non-evidence-based medicine cannot immediately see that giving a patient this substance diluted in 10^20 times as much water or sticking a needle in this particular spot or whatever will help cure the patient. (Likewise, musicians are normally evidence-based musicians to some extent, but astrologists are not evidence-based astrologists; can you find more examples?)
If you interpret evidence-based in the widest sense possible, the phrase sort of loses its meaning. Note that the very post you quote explains the intended contrast between systematic and statistical use of evidence versus intuition and traditional experience based human learning.
Besides, would you not say that astrologers figure out both how to be optimally vague, avoiding being wrong while exciting their readers, much the same way musicians figure out what sounds good?
Yes, but “intuition and traditional experience based human learning” is probably much less reliable in medicine than it is in barbering, so the latter isn’t a good example in a discussion about the former.
:-)
Something similar could be said about practitioners of alternative medicine, though.
The goal of barbering is to create haircuts that increase the attractiveness of the client to people besides the barber and the client.
A barber might think: “All my clients look really great”, when in reality his haircuts reduce the attractiveness of the clients.
Surely, judging someone’s attractiveness using your System 1 alone is less hard than judging someone’s health using your System 1 alone, for most people in most situations?
A professional barber is likely to notice a lot of things about a haircut that the average person doesn’t see. It could be that he creates haircuts that look impressive to other barbers but don’t look good to the average person of the opposing sex who isn’t a barber.
I do think that you can get a decent assessment of someone’s backpain by asking them whether it has gotten better. Actually that’s even how most scientific studies who measure pain do it. They let the person rate their pain subjectively and when the subjective rating gets better through the drug they see it as a win.
For a lot of serious health issues it’s easy to see when a person gets better.
Most homeopathists spend more time interviewing their patients and getting a good understanding of their condition than the average mainstream doctor who takes 5 minutes per patient.
I think the barbering example is excellent—it illustrates that, while controlled experiments more or less is physics, and while physics is great, it is probably not going to bring a paradigm shift to barbering any time soon. One should not expect all domains to be equally well suited to a cut and dried scientific approach.
Where medicine lies on this continuum of suitedness is an open question—it is probably even a misleading question, with medicine being a collection of vastly different problems. However, it is not at all obvious that simply turning up the scientificness dial is going to make things better. It is for instance conceivable that there are already people treating medicine as a hard science, and that the current balance of intuition and evidence in medicine reflects how effective these two approaches are.
I am not trying to argue whether astrology is evidence-based or not. I am saying that the very inclusive definition of evidence-based which encompasses barbering is, (a) nearly useless because it includes every possible way of doing medicine and (b) probably not the one intended by the others using the term.
Huh? What evidence are homoeopathy and crystal healing and similar (assuming that’s what Qiaochu_Yuan meant by “other kinds”) based on?
EDIT: Apparently not.
“Other kinds” meant “whatever mainstream medicine does that doesn’t fall under the evidence-based label,” not alternative medicine. I should’ve been clearer.
Yes, I realized that later, while reading another branch of the thread (see my edit).
What do you mean with “mainstream medicine” in that context?
What ambiguity is there in what I mean by “mainstream medicine” here?
A lot of people use the term evidence-based medicine interchangeable with mainstream medicine. What’s in your opinion medicine that counts as mainstream medicine but that doesn’t count as evidence-based medicine?
That doesn’t agree with my experience. Evidence-based medicine refers to a specific and recent movement within mainstream medicine, which is much older.
If a doctor would today practice medicine the exact way it was practiced in 1950 I don’t think you would say that the doctor practices mainstream medicine.
If you define “mainstream” by the amount of people who use it than homeopathy is probably “mainstream medicine”. Even if you go by the status of the people, when the Queen uses homeopathy it’s no low status treatment.
There are two reasons why homeopathy gets classified as “alternative medicine”.
(1) It’s uses a ideological framework that goes against the reductionist world view.
(2) There’s are not enough high quality double blind studies to allow an institution such as cochrane to recommend homeopathy as a treatment.
The term Evidence-Based medicine got made up by a bunch of university professors in 1992 to describe the style of medicine that they were teaching. At the beginning the term intentionally downplayed clinicial experience. Today most medicial schools say that they teach Evidence-Based medicine but they weakened the definition in a way that allows clinicians using their clinical experience but that still focuses on peer reviewed trials.
If you don’t pracitice medicine the way the university teach it in their normal programs than you are practicing in my opinion “alternative medicine”.
There are even dozens of scientific studies that support homeopathy. According to a report titled “Effectiveness, Safety and Cost-Effectiveness of Homeopathy in General Practice – Summarized Health Technology Assessment” commissioned by the Swiss government:
There are plenty people out there who can explain you why all those homeopathy studies are flawed, but on the other hand how many double blind controlled trials do you know that show that barbers can create haircuts that increase someone’s chances with the opposing sex?
But in general people do buy homeopathic medicine not because they read the report of the Swiss government and belief it. They buy it based on anecdotal evidence. They hear that some friend had success with homeopathy and then the go out and buy it.
The fact that you are ideologically opposed to homeopathy and crystal healing working, doesn’t mean that it fails to produce anecdotal evidence.
That’s wrong. If a acupuncturist puts needles in 10 people and 5 of them lose their back pain than he has “unsystematic clinical experience” that provides evidence for his treatment.
The core of evidence-based medicine is the belief that you shouldn’t use that kind of evidence for clinical decision making but that doctors should read medicial journals that report clinical trials that show whether or not a treatment works.
Actually musicians and astrologists are very similar. Both make money with providing entertaining performances for their clients. Members of those professions who ignore evidence about what entertains their clients go out of business.
Maybe some of those 5 would have lost their pain even without needles. Whereas the barber knows what his client would have looked like without the hair cut.
Right, that’s why it’s unsystematic.
In the Bayesian sense of the word, “I stuck a needle in this person and the amount of pain he reported went down” would have to be considered to be evidence that would increase the Bayesian possibility that your hypothesis that acupuncture helps back pain is correct. However, it’s not systematic, scientific evidence. To get that kind of evidence, you would have to do systematic studies of a large number of people, give some of them acupuncture and give some of them asprin, and see what the statistical result is.
I think that’s what bogging this discussion down here, is that the word “evidence” is being used in two different ways. If we were perfectly rational beings, we would be able to use either kind of evidence, but the problem is that the first kind of evidence (individual unsystematic personal experiences) tends to be warped by all kinds of biases (selection bias, especially) making it hard to use in any kind of reliable way. You use it if it’s all you have, but systematic evidence is very much preferable.
OK, if you consider the point of astrology to be “making money”, as opposed to “predicting people’s personalities and future events”, then it is evidence-based—but then again, if you consider the point of alternative medicine to be “making money”, as opposed to “improving people’s health”, then it is evidence-based as well. (But now that Qiaochu_Yuan has made clear that it’s not alternative medicine that he was talking about, this is kind of moot, so I’ll tap out now.)
I didn’t. I advocated another goal, entertainment. I don’t know that much about astrology but I think a fair percentage of the people who do pay a astrologists do it for entertainement purposes.
Letting someone stick needles inside you, when you go to a acupuncturist is less about getting entertainement.
The kind of people who like astrology often also like other personality tests that they find in magazines. People enjoy going through those tests.
If an astrologer would tell people something about their personality that’s accurate but that those people aren’t willing to accept, I doubt he would stay long in business.
A bit like the musician who only plays music that he himself considers to be good, but that’s “too advanced” for his audience. If the musician only sees his own opinion of his work he’s not different than an astrologer who only sees whether his horoscope is good. If you call that musician “evidence-based” than the astrologer who goes after his own judgement of his work is also “evidence-based”.
Why does that matter to the question whether barbers can be meaningfully to be said to practice evidence-based barbering?
I was claiming that barbering is more evidence-based than alternative medicine, but if alternative medicine is not what’s being discussed, then even if I turned out to be right it still wouldn’t be relevant.