I don’t see there as being a ‘fundamental confusion’ here, and not even that much of a fundamental disagreement.
When I crunched the numbers on ‘how much good do doctors do’ it was meant to provide a rough handle on a plausible upper bound: even if we beg the question against critics of medicine (of which there are many), and even if we presume any observational marginal response is purely causal (and purely mediated by doctors), the numbers aren’t (in EA terms) that exciting in terms of direct impact.
In talks, I generally use the upper 95% confidence bound or central estimate of the doctor coefficient as a rough steer (it isn’t a significant predictor, and there’s reasonable probability mass on the impact being negative): although I suspect there will be generally unaccounted confounders attenuating ‘true’ effect rather than colliders masking it, these sort of ecological studies are sufficiently insensitive to either to be no more than indications—alongside the qualitative factors—that the ‘best (naive) case’ for direct impact as a doctor isn’t promising.
There’s little that turns on which side of zero our best guess falls, so long as we be confident it is a long way down from the best candidates: on the scale of intervention effectiveness, there’s not that much absolute distance between estimates (I suspect) Hanson or I would offer. There might not be much disagreement even in coarse qualitative terms: Hanson’s work here—I think—focuses on the US, and US health outcomes are a sufficiently pathological outlier in the world I’m also unsure whether marginal US medical effort is beneficial; I’m not sure Hanson has staked out a view on whether he’s similarly uncertain about positive marginal impact in non-US countries, so he might agree with my view it is (modestly) net-positive, despite its dysfunction (neither I nor what I wrote assumes the system ‘basically knows what it’s doing’ in the common-sense meaning).
If Hanson has staked out this broader view, then I do disagree with it, but I don’t think this disagreement would indicate at least one of us has to be ‘deeply confused’ (this looks like a pretty crisp disagreement to me) nor ‘badly misinformed’ (I don’t think there are key considerations one-or-other of us is ignorant of which explains why one of us errs to sceptical or cautiously optimistic). My impressions are also less sympathetic to ‘signalling accounts’ of healthcare than his (cf.) - but again, my view isn’t ‘This is total garbage’, and I doubt he’s monomaniacally hedgehog-y about the signalling account. (Both of us have also argued for attenuating our individual impressions in deference to a wider consensus/outside view for all things considered judgements).
Although I think the balance of expertise leans against archly sceptical takes on medicine, I don’t foresee convincing adjudication on this point coming any time soon, nor that EA can reasonably expect to be the ones to provide this breakthrough—still less for all the potential sign-inverting crucial considerations out there. Stumbling on as best we can with our best guess seems a better approach than being paralyzed until we’re sure we’ve figured it all out.
Something that nets out to a small or no effect because large benefits and harms cancel out is very different (with different potential for impact) than something like, say, faith healing, where you can’t outperform just by killing fewer patients. A marginalist analysis that assumes that the person making the decision doesn’t know their own intentions & is just another random draw of a ball from an urn totally misses this factor.
A marginalist analysis that assumes that the person making the decision doesn’t know their own intentions & is just another random draw of a ball from an urn totally misses this factor.
Happily, this factor has not been missed by either my profile or 80k’s work here more generally. Among other things, we looked at:
Variance in impact between specialties and (intranational) location (1) (as well as variance in earnings for E2G reasons) (2, also, cf.)
Areas within medicine which look particularly promising (3)
Why ‘direct’ clinical impact (either between or within clinical specialties) probably has limited variance versus (e.g.) research (4), also
I also cover this in talks I have given on medical careers, as well as when offering advice to people contemplating a medical career or how to have a greater impact staying within medicine.
I still think trying to get a handle on the average case is a useful benchmark.
[I wrote the 80k medical careers page]
I don’t see there as being a ‘fundamental confusion’ here, and not even that much of a fundamental disagreement.
When I crunched the numbers on ‘how much good do doctors do’ it was meant to provide a rough handle on a plausible upper bound: even if we beg the question against critics of medicine (of which there are many), and even if we presume any observational marginal response is purely causal (and purely mediated by doctors), the numbers aren’t (in EA terms) that exciting in terms of direct impact.
In talks, I generally use the upper 95% confidence bound or central estimate of the doctor coefficient as a rough steer (it isn’t a significant predictor, and there’s reasonable probability mass on the impact being negative): although I suspect there will be generally unaccounted confounders attenuating ‘true’ effect rather than colliders masking it, these sort of ecological studies are sufficiently insensitive to either to be no more than indications—alongside the qualitative factors—that the ‘best (naive) case’ for direct impact as a doctor isn’t promising.
There’s little that turns on which side of zero our best guess falls, so long as we be confident it is a long way down from the best candidates: on the scale of intervention effectiveness, there’s not that much absolute distance between estimates (I suspect) Hanson or I would offer. There might not be much disagreement even in coarse qualitative terms: Hanson’s work here—I think—focuses on the US, and US health outcomes are a sufficiently pathological outlier in the world I’m also unsure whether marginal US medical effort is beneficial; I’m not sure Hanson has staked out a view on whether he’s similarly uncertain about positive marginal impact in non-US countries, so he might agree with my view it is (modestly) net-positive, despite its dysfunction (neither I nor what I wrote assumes the system ‘basically knows what it’s doing’ in the common-sense meaning).
If Hanson has staked out this broader view, then I do disagree with it, but I don’t think this disagreement would indicate at least one of us has to be ‘deeply confused’ (this looks like a pretty crisp disagreement to me) nor ‘badly misinformed’ (I don’t think there are key considerations one-or-other of us is ignorant of which explains why one of us errs to sceptical or cautiously optimistic). My impressions are also less sympathetic to ‘signalling accounts’ of healthcare than his (cf.) - but again, my view isn’t ‘This is total garbage’, and I doubt he’s monomaniacally hedgehog-y about the signalling account. (Both of us have also argued for attenuating our individual impressions in deference to a wider consensus/outside view for all things considered judgements).
Although I think the balance of expertise leans against archly sceptical takes on medicine, I don’t foresee convincing adjudication on this point coming any time soon, nor that EA can reasonably expect to be the ones to provide this breakthrough—still less for all the potential sign-inverting crucial considerations out there. Stumbling on as best we can with our best guess seems a better approach than being paralyzed until we’re sure we’ve figured it all out.
Something that nets out to a small or no effect because large benefits and harms cancel out is very different (with different potential for impact) than something like, say, faith healing, where you can’t outperform just by killing fewer patients. A marginalist analysis that assumes that the person making the decision doesn’t know their own intentions & is just another random draw of a ball from an urn totally misses this factor.
Happily, this factor has not been missed by either my profile or 80k’s work here more generally. Among other things, we looked at:
Variance in impact between specialties and (intranational) location (1) (as well as variance in earnings for E2G reasons) (2, also, cf.)
Areas within medicine which look particularly promising (3)
Why ‘direct’ clinical impact (either between or within clinical specialties) probably has limited variance versus (e.g.) research (4), also
I also cover this in talks I have given on medical careers, as well as when offering advice to people contemplating a medical career or how to have a greater impact staying within medicine.
I still think trying to get a handle on the average case is a useful benchmark.
I just want to register disagreement.