I got to shadow some breast physicians last month, and although it’s sort of off topic I think I gained some insight as to why so many doctors get this question wrong.
Which is because it’s very different from any situation they ever come across in clinical practice. Guidelines are to screen people with mammography and examination; anyone who comes up as suspicious on those two tests then gets a biopsy. No one gets diagnosed with breast cancer from a mammogram alone, the progression from mammogram on to the next step is hard-coded into a pre-determined algorithm, and so the question of “This woman got a positive on the mammogram; does she have cancer?” never comes up. A question that does come up a lot is a woman panicking because she got a positive mammogram and demanding to know whether she has breast cancer, and the inevitable answer is “We’ll need to do more tests, but don’t worry too much yet because most of these things are false positives.”
So the doctors involved know that most real mammogram results are false positives, they know how to diagnose breast cancer based on the combination of tests they actually do, they just can’t do Bayesian math problems when given probabilities. This is kind of interesting if you’re curious about their intelligence but as far as I know doesn’t really affect clinical care.
As far as the take-home practical message goes, on my reading it was never about how well doctors could “diagnose cancer” per se based on mammogram results—rather, the reason we ask about P(cancer | positive) is because it ought to inform our decision about whether a biopsy is really warranted. If a healthy young woman from a population with an exceedingly low base rate for breast cancer has a positive mammogram, the prior probability of her having cancer may still be low enough that there might actually be negative expected value in following up with a biopsy; after all, let’s not forgot that a biopsy is not a trivial procedure and things do sometimes go wrong.
So I think this actually does have some implication for real-world clinical care: we ought to question whether it is wise to automatically follow up all positive mammograms with biopsies. Maybe it is, and maybe it isn’t, but I don’t think we should take the question for granted as appears to be the case.
If a biopsy is the next step in diagnosing breast cancer after a positive mammogram, then we shouldn’t perform mammograms on anyone it still wouldn’t be worth biopsying should their mammogram turn up positive.
And although I’m having a hard time finding a news article to verify this, someone informed me that the official breast cancer screening recommendations in the US (or was it a particular state, perhaps California?) were recently modified such that it is now not recommended that women younger than 40 (50?) receive regular screening. The young woman who informed me of this change in policy was quite upset about it. It didn’t make any sense to her. I tried to explain to her how it actually made good sense when you think about it in terms of base rates and expected values, but of course, it was no use.
But to return to the issue clinical implications, yes: if a woman belongs to a population where the result of a mammogram would not change our decision about whether a biopsy is necessary, then probably she shouldn’t have the mammogram. I suspect that this line of reasoning would sound quite foreign to most practicing doctors.
Now that’s some interesting back story. I could see that if one knew that the route from positive test through to establishing conclusively if cancer is present was a fluid path, one might not be overly concerned with the test result itself.
As to this specific problem, I just used what EY used; perhaps there are more applicable/pertinent statistics problems that could be used.
I’ve been trying to think of other visualization tools that might be more universal or intuitive. I get the sliding java applets, but think if one can tie what they’re showing to a real world tool or process of some sort, it will help. What these are doing is no different than his. The “top bar” are the two original spheres. The “bottom bar” is the reduced amount of each sphere (0.8 x 0.01 and 0.096 x 0.99) that remains after sifting.
I got to shadow some breast physicians last month, and although it’s sort of off topic I think I gained some insight as to why so many doctors get this question wrong.
Which is because it’s very different from any situation they ever come across in clinical practice. Guidelines are to screen people with mammography and examination; anyone who comes up as suspicious on those two tests then gets a biopsy. No one gets diagnosed with breast cancer from a mammogram alone, the progression from mammogram on to the next step is hard-coded into a pre-determined algorithm, and so the question of “This woman got a positive on the mammogram; does she have cancer?” never comes up. A question that does come up a lot is a woman panicking because she got a positive mammogram and demanding to know whether she has breast cancer, and the inevitable answer is “We’ll need to do more tests, but don’t worry too much yet because most of these things are false positives.”
So the doctors involved know that most real mammogram results are false positives, they know how to diagnose breast cancer based on the combination of tests they actually do, they just can’t do Bayesian math problems when given probabilities. This is kind of interesting if you’re curious about their intelligence but as far as I know doesn’t really affect clinical care.
As far as the take-home practical message goes, on my reading it was never about how well doctors could “diagnose cancer” per se based on mammogram results—rather, the reason we ask about P(cancer | positive) is because it ought to inform our decision about whether a biopsy is really warranted. If a healthy young woman from a population with an exceedingly low base rate for breast cancer has a positive mammogram, the prior probability of her having cancer may still be low enough that there might actually be negative expected value in following up with a biopsy; after all, let’s not forgot that a biopsy is not a trivial procedure and things do sometimes go wrong.
So I think this actually does have some implication for real-world clinical care: we ought to question whether it is wise to automatically follow up all positive mammograms with biopsies. Maybe it is, and maybe it isn’t, but I don’t think we should take the question for granted as appears to be the case.
If a biopsy is the next step in diagnosing breast cancer after a positive mammogram, then we shouldn’t perform mammograms on anyone it still wouldn’t be worth biopsying should their mammogram turn up positive.
Yes, that’s exactly right.
And although I’m having a hard time finding a news article to verify this, someone informed me that the official breast cancer screening recommendations in the US (or was it a particular state, perhaps California?) were recently modified such that it is now not recommended that women younger than 40 (50?) receive regular screening. The young woman who informed me of this change in policy was quite upset about it. It didn’t make any sense to her. I tried to explain to her how it actually made good sense when you think about it in terms of base rates and expected values, but of course, it was no use.
But to return to the issue clinical implications, yes: if a woman belongs to a population where the result of a mammogram would not change our decision about whether a biopsy is necessary, then probably she shouldn’t have the mammogram. I suspect that this line of reasoning would sound quite foreign to most practicing doctors.
Now that’s some interesting back story. I could see that if one knew that the route from positive test through to establishing conclusively if cancer is present was a fluid path, one might not be overly concerned with the test result itself.
As to this specific problem, I just used what EY used; perhaps there are more applicable/pertinent statistics problems that could be used.
I’ve been trying to think of other visualization tools that might be more universal or intuitive. I get the sliding java applets, but think if one can tie what they’re showing to a real world tool or process of some sort, it will help. What these are doing is no different than his. The “top bar” are the two original spheres. The “bottom bar” is the reduced amount of each sphere (0.8 x 0.01 and 0.096 x 0.99) that remains after sifting.
Just a different way to look at it.