There’s a vaccine that purports to reduce the risk of a certain infection… some of the time.
After doing some investigation, you come up with the following statistics:
Roughly 10% of the population has taken the vaccine.
Of those that have taken the vaccine, the infection rate is 25%.
Of those that have not taken the vaccine, the infection rate is 10%.
P(Infection|vaccine) > P(Infection|~vaccine), so EDT says don’t give your child the vaccine.
You do some more research and discover the following:
The infection rate for unvaccinated heterosexuals is 11%. The infection rate for vaccinated heterosexuals is 9%. The infection rate for unvaccinated non-heterosexuals is 40%. The infection rate for vaccinated non-heterosexuals is 30%. Non-heterosexuals are a small minority of the population, but are disproportionally represented among all those who are vaccinated. Getting vaccinated doesn’t change a person’s sexual orientation.
EDT now comes to the ridiculous conclusion that heterosexuals should take the vaccine and that non-heterosexuals should take the vaccine, but people whose sexual orientation is unknown should not—even though everyone is either heterosexual or non-heterosexual.
I don’t think this math is right. There are two general problems I see with it. Number one is that once you find out this,
Non-heterosexuals are a small minority of the population, but are disproportionally represented among all those who are vaccinated.
your P(infection|vaccine) and P(infection|~vaccine) change to values that make more intuitive sense.
Number two is that I don’t think the percentages you gave in the problem are actually coherent. I don’t believe it’s possible to get the 25% infected and vaccinated and 10% infected and unvaccinated with the numbers provided, no matter what values you use for the rate of heterosexuality in the population, and in the vaccinated and unvaccinated groups. I don’t really want to discuss that in detail, so for the sake of my post here, I’m going to say that 80% of the population is heterosexual, and that 85% of the individuals who have gotten the vaccine are non-heterosexual.
Using these numbers, I calculate that the initial study would find that 26.85% of the vaccinated individuals are infected, and 14.70555...% of the unvaccinated individuals are infected. Note that given just this information, these are the probabilities that a bayesian should assign to the proposition(s), “If I (do not) get the vaccine, I will become infected”. (Unless of course said bayesian managed to pay attention enough to realize that this wasn’t a randomized, controlled experiment, and that therefore the results are highly suspect).
However, once we learn that this survey wasn’t performed using proper, accurate scientific methods, and we gather more data (presumably paying a bit more attention to the methodology), we can calculate a new P(infected|vaccinated) and a new P(infected|unvaccinated) for our child of unknown sexual orientation. As I calculate it, if you believe that your child is heterosexual with 80% confidence (which is the general rate of heterosexuality in the population, in our hypothetical scenario), you calculate that P(infection|vaccinated) = .132, and P(infection|unvaccinated) = .168. So, EDT says to get the vaccine. Alternatively, let’s say you’re only 10% confident your child is heterosexual. In this case, P(infection|vaccinated) is .279, and P(infection|unvaccinated) = .371. Definitely get the vaccine. Say you’re 90% confident your child is heterosexual. Then P(infection|vaccinated) = .111, while P(infection|unvaccinated) = .139. Still, get the vaccine.
Ultimately, using the raw data from the biased study cited initially as your actual confidence level makes about as much sense as applying Laplace’s rule of succession in a case where you know the person drawing the balls is searching through the bag to draw out only the red balls, and concluding that if you draw out a ball from the bag without looking, it will almost certainly be red. It’s simply the wrong way for a bayesian to calculate the probability.
If anything, I think this hypothetical scenario is not so much a refutation of EDT, so much as a demonstration of why proper scientific methodology is important.
Okay, this is going to sound weird. But again, how do I know that getting vaccinated doesn’t change a person’s sexual orientation? Presumably, the drug was tested, and someone would have noticed if the intervention group turned out “less straight” than the controls. But that doesn’t rule out the possibility that something about me choosing to have my child vaccinated will make him/her non-heterosexual. To us, this just contradicts common sense about how the world works, but that common sense isn’t represented anywhere in the problem as you stated it.
If all I know about the universe I inhabit are the conditional probabilities you gave, then no, I wouldn’t have my child vaccinated. In fact, I would even have to precommit to not having my child vaccinated, in the case that I find out his/her sexual orientation later. On the face of it, this conclusion isn’t any more ridiculous than Prisoner’s dilemma, where you should defect if your opponent defects, and defect if your opponent cooperates, but should cooperate if you don’t know what your opponent will do. Even though your opponent will either cooperate or defect.
UDT solves the apparent contradiction by saying that you should cooperate even if you know what your opponent will choose (as long as you think your opponent is enough like you that you would have predicted his decision to correlate with your own, had you not already known his decision). But this also leads to an intuitively absurd conclusion: that you should care about all worlds that could have been the same way you care about your own world. Sacrificing your own universe for the sake of a universe that could have been may be ethical, in the very broadest sense of altruism, but it doesn’t seem rational.
No, lower risk people seek out the vaccine: it’s right there, people who are vaccinated are less likely to be infected, just like people who are less likely to get lung cancer eschew smoking.
This example is isomorphic to the Smoker’s lesion, with being non-heterosexual = having the lesion, infection = cancer, taking the vaccine = stopping smoking. The recipe suggested in the original post can be extended here: separate the decision to take the vaccine using EDT while not knowing about own sexual orientation (A), the state of having taken the vaccine (E) and the outcome of not being infected (O). Although P(O|E) < P(O|~E), it is not true that P(O|A) < P(O|~A).
EDT chokes on Simpson’s Paradox in general.
Consider:
There’s a vaccine that purports to reduce the risk of a certain infection… some of the time.
After doing some investigation, you come up with the following statistics:
Roughly 10% of the population has taken the vaccine. Of those that have taken the vaccine, the infection rate is 25%. Of those that have not taken the vaccine, the infection rate is 10%.
P(Infection|vaccine) > P(Infection|~vaccine), so EDT says don’t give your child the vaccine.
You do some more research and discover the following:
The infection rate for unvaccinated heterosexuals is 11%.
The infection rate for vaccinated heterosexuals is 9%.
The infection rate for unvaccinated non-heterosexuals is 40%.
The infection rate for vaccinated non-heterosexuals is 30%.
Non-heterosexuals are a small minority of the population, but are disproportionally represented among all those who are vaccinated.
Getting vaccinated doesn’t change a person’s sexual orientation.
EDT now comes to the ridiculous conclusion that heterosexuals should take the vaccine and that non-heterosexuals should take the vaccine, but people whose sexual orientation is unknown should not—even though everyone is either heterosexual or non-heterosexual.
I don’t think this math is right. There are two general problems I see with it. Number one is that once you find out this,
your P(infection|vaccine) and P(infection|~vaccine) change to values that make more intuitive sense.
Number two is that I don’t think the percentages you gave in the problem are actually coherent. I don’t believe it’s possible to get the 25% infected and vaccinated and 10% infected and unvaccinated with the numbers provided, no matter what values you use for the rate of heterosexuality in the population, and in the vaccinated and unvaccinated groups. I don’t really want to discuss that in detail, so for the sake of my post here, I’m going to say that 80% of the population is heterosexual, and that 85% of the individuals who have gotten the vaccine are non-heterosexual.
Using these numbers, I calculate that the initial study would find that 26.85% of the vaccinated individuals are infected, and 14.70555...% of the unvaccinated individuals are infected. Note that given just this information, these are the probabilities that a bayesian should assign to the proposition(s), “If I (do not) get the vaccine, I will become infected”. (Unless of course said bayesian managed to pay attention enough to realize that this wasn’t a randomized, controlled experiment, and that therefore the results are highly suspect).
However, once we learn that this survey wasn’t performed using proper, accurate scientific methods, and we gather more data (presumably paying a bit more attention to the methodology), we can calculate a new P(infected|vaccinated) and a new P(infected|unvaccinated) for our child of unknown sexual orientation. As I calculate it, if you believe that your child is heterosexual with 80% confidence (which is the general rate of heterosexuality in the population, in our hypothetical scenario), you calculate that P(infection|vaccinated) = .132, and P(infection|unvaccinated) = .168. So, EDT says to get the vaccine. Alternatively, let’s say you’re only 10% confident your child is heterosexual. In this case, P(infection|vaccinated) is .279, and P(infection|unvaccinated) = .371. Definitely get the vaccine. Say you’re 90% confident your child is heterosexual. Then P(infection|vaccinated) = .111, while P(infection|unvaccinated) = .139. Still, get the vaccine.
Ultimately, using the raw data from the biased study cited initially as your actual confidence level makes about as much sense as applying Laplace’s rule of succession in a case where you know the person drawing the balls is searching through the bag to draw out only the red balls, and concluding that if you draw out a ball from the bag without looking, it will almost certainly be red. It’s simply the wrong way for a bayesian to calculate the probability.
If anything, I think this hypothetical scenario is not so much a refutation of EDT, so much as a demonstration of why proper scientific methodology is important.
Well, you’re right that I did make up the numbers without checking anything. :(
Here’s a version with numbers that work, courtesy of Judea Pearl’s book: http://escholarship.org/uc/item/3s62r0d6
Okay, this is going to sound weird. But again, how do I know that getting vaccinated doesn’t change a person’s sexual orientation? Presumably, the drug was tested, and someone would have noticed if the intervention group turned out “less straight” than the controls. But that doesn’t rule out the possibility that something about me choosing to have my child vaccinated will make him/her non-heterosexual. To us, this just contradicts common sense about how the world works, but that common sense isn’t represented anywhere in the problem as you stated it.
If all I know about the universe I inhabit are the conditional probabilities you gave, then no, I wouldn’t have my child vaccinated. In fact, I would even have to precommit to not having my child vaccinated, in the case that I find out his/her sexual orientation later. On the face of it, this conclusion isn’t any more ridiculous than Prisoner’s dilemma, where you should defect if your opponent defects, and defect if your opponent cooperates, but should cooperate if you don’t know what your opponent will do. Even though your opponent will either cooperate or defect.
UDT solves the apparent contradiction by saying that you should cooperate even if you know what your opponent will choose (as long as you think your opponent is enough like you that you would have predicted his decision to correlate with your own, had you not already known his decision). But this also leads to an intuitively absurd conclusion: that you should care about all worlds that could have been the same way you care about your own world. Sacrificing your own universe for the sake of a universe that could have been may be ethical, in the very broadest sense of altruism, but it doesn’t seem rational.
In this case, it’s because I said so. ;)
We could imagine another study where this was observed.
EDT can’t come to the conclusion that getting vaccinated doesn’t cause, or share a common cause with, homosexuality.
Here, your premise is actually that homosexuality causes vaccinations.
Exactly. They know they’re at higher risk, so they’re more likely to seek out the vaccine.
No, lower risk people seek out the vaccine: it’s right there, people who are vaccinated are less likely to be infected, just like people who are less likely to get lung cancer eschew smoking.
This example is isomorphic to the Smoker’s lesion, with being non-heterosexual = having the lesion, infection = cancer, taking the vaccine = stopping smoking. The recipe suggested in the original post can be extended here: separate the decision to take the vaccine using EDT while not knowing about own sexual orientation (A), the state of having taken the vaccine (E) and the outcome of not being infected (O). Although P(O|E) < P(O|~E), it is not true that P(O|A) < P(O|~A).
Your example contains several unstated assumptions:
People know their sexual orientation
The decision to take the vaccine is voluntary
People base that decision on their sexual orientation
Hence the decision to take the vaccine with an unknown sexual orientation never occours.
EDT chokes because it ignores the obvious extra controlling principle: those with no sexual experience don’t generally have the infection.
Add that, and you are fine. Vaccinate before the time of first expected exposure