Well, I hate to say this for obvious reasons, but if the magic sugar water cured my hayfever just once, I’d try it again, and if it worked again, I’d try it again.
(This tripped my positive bias sense: only testing the outcome in the presence of an intervention doesn’t establish that it’s doing anything. It’s wrong to try again and again after something seemed to work, one should also try not doing it and see if it stops working. Scattering anti-tiger pills around town also “works”: if one does that every day, there will be no tigers in the neighborhood.)
Scattering anti-tiger pills around town also “works”: if one does that every day, there will be no tigers in the neighborhood.
That’s a bad analogy. If “anti-tiger pills” repeatedly got rid of a previously observed real tiger, you would be well advised to give the issue some thought.
That’s a bad analogy. If “anti-tiger pills” repeatedly got rid of a previously observed real tiger, you would be well advised to give the issue some thought.
What’s that line about how, if you treat a cold, you can get rid of it in seven days, but otherwise it lasts a week?
You would still want to check to see whether tigers disappear even when no “anti-tiger pills” are administered.
You would still want to check to see whether tigers disappear even when no “anti-tiger pills” are administered.
Depending on how likely the tiger is to eat people if it didn’t disappear, and your probability of the pills successfully repelling the tiger given that it’s always got rid of the tiger, and the cost of the pill and how many people the tiger is expected to eat if it doesn’t disappear? Not always.
A medical example of this is the lack of evidence for the efficacy of antihistamine against anaphylaxis. When I asked my sister (currently going through clinical school) about why, she said “because if you do a study, people in the control group will die if these things work, and we have good reason to believe they do”
and your probability of the pills successfully repelling the tiger given that it’s always got rid of the tiger
Yes. But the point is that this number should be negligible if you haven’t seen how the tiger behaves in the absence of the pills. (All of this assumes that you do not have any causal model linking pill-presence to tiger-absence.)
This case differs from the use of antihistamine against anaphylaxis for two reasons:
There is some theoretical reason to anticipate that antihistamine would help against anaphylaxis, even if the connection hasn’t been nailed down with double-blind experiments.
We have cases where people with anaphylaxis did not receive antihistamine, so we can compare cases with and without antihistamine. The observations might not have met the rigorous conditions of a scientific experiment, but that is not necessary for the evidence to be rational and to justify action.
Absolutely. The precise thing that matters is the probability tigers happen if you don’t use the pills. So, say, I wouldn’t recommend doing the experiment if you live in areas with high densities of tigers (which you do if there’s one showing up every day!) and you weren’t sure what was going into the pills (tiger poison?), but would recommend doing the experiment if you lived in London and knew that the pills were just sugar.
Similarly, I’m more likely to just go for a herbal remedy that hasn’t had scientific testing, but has lots of anecdotal evidence for lack of side-effects, than a homeopathic remedy with the same amount of recommendation.
It is positive bias (in that this isn’t the best way to acquire knowledge), but there’s a secondary effect: the value of knowing whether or not the magic sugar water cures his hayfever is being traded off against the value of not having hayfever.
Depending on how frequently he gets hayfever, and how long it took to go away without magic sugar water, and how bothersome it is, and how costly the magic sugar water is, it may be better to have an unexplained ritual for that portion of his life than to do informative experiments.
(And, given that the placebo effect is real, if he thinks the magic sugar water is placebo, that’s reason enough to drink it without superior alternatives.)
Agree with this. Knowing the truth has a value and a cost (doing the experiment).
I recently heard something along the lines of: “We don’t have proof that antihistamines work to treat anaphylaxis, because we haven’t done the study. But the reason we haven’t done the study is because we’re pretty sure the control group would die.”
I agree, I’d try not taking it too! I had hayfever as a child, and it was bloody awful. I used to put onionjuice in my eyes because it was the only thing that would provide relief. But even as a child I was curious enough to try it both ways.
(This tripped my positive bias sense: only testing the outcome in the presence of an intervention doesn’t establish that it’s doing anything. It’s wrong to try again and again after something seemed to work, one should also try not doing it and see if it stops working. Scattering anti-tiger pills around town also “works”: if one does that every day, there will be no tigers in the neighborhood.)
That’s a bad analogy. If “anti-tiger pills” repeatedly got rid of a previously observed real tiger, you would be well advised to give the issue some thought.
What’s that line about how, if you treat a cold, you can get rid of it in seven days, but otherwise it lasts a week?
You would still want to check to see whether tigers disappear even when no “anti-tiger pills” are administered.
Depending on how likely the tiger is to eat people if it didn’t disappear, and your probability of the pills successfully repelling the tiger given that it’s always got rid of the tiger, and the cost of the pill and how many people the tiger is expected to eat if it doesn’t disappear? Not always.
A medical example of this is the lack of evidence for the efficacy of antihistamine against anaphylaxis. When I asked my sister (currently going through clinical school) about why, she said “because if you do a study, people in the control group will die if these things work, and we have good reason to believe they do”
EDIT: I got beaten to posting this by the only other person I told about it
Yes. But the point is that this number should be negligible if you haven’t seen how the tiger behaves in the absence of the pills. (All of this assumes that you do not have any causal model linking pill-presence to tiger-absence.)
This case differs from the use of antihistamine against anaphylaxis for two reasons:
There is some theoretical reason to anticipate that antihistamine would help against anaphylaxis, even if the connection hasn’t been nailed down with double-blind experiments.
We have cases where people with anaphylaxis did not receive antihistamine, so we can compare cases with and without antihistamine. The observations might not have met the rigorous conditions of a scientific experiment, but that is not necessary for the evidence to be rational and to justify action.
Absolutely. The precise thing that matters is the probability tigers happen if you don’t use the pills. So, say, I wouldn’t recommend doing the experiment if you live in areas with high densities of tigers (which you do if there’s one showing up every day!) and you weren’t sure what was going into the pills (tiger poison?), but would recommend doing the experiment if you lived in London and knew that the pills were just sugar.
Similarly, I’m more likely to just go for a herbal remedy that hasn’t had scientific testing, but has lots of anecdotal evidence for lack of side-effects, than a homeopathic remedy with the same amount of recommendation.
It is positive bias (in that this isn’t the best way to acquire knowledge), but there’s a secondary effect: the value of knowing whether or not the magic sugar water cures his hayfever is being traded off against the value of not having hayfever.
Depending on how frequently he gets hayfever, and how long it took to go away without magic sugar water, and how bothersome it is, and how costly the magic sugar water is, it may be better to have an unexplained ritual for that portion of his life than to do informative experiments.
(And, given that the placebo effect is real, if he thinks the magic sugar water is placebo, that’s reason enough to drink it without superior alternatives.)
Agree with this. Knowing the truth has a value and a cost (doing the experiment).
I recently heard something along the lines of: “We don’t have proof that antihistamines work to treat anaphylaxis, because we haven’t done the study. But the reason we haven’t done the study is because we’re pretty sure the control group would die.”
I agree, I’d try not taking it too! I had hayfever as a child, and it was bloody awful. I used to put onion juice in my eyes because it was the only thing that would provide relief. But even as a child I was curious enough to try it both ways.