What Alex Tabarrok called The Paxlovid Paradox is getting noticed by the people who notice such things, and completely ignored by everyone else. I’ve split off this week’s Paxlovid update to help make the situation easier to notice, and easier to remember and reference later, lest we forget.
Supply and Demand
The good news is that we are confident Paxlovid is safe and effective and the purchasing department is acting accordingly. Once we decide it is legal, we’ve secured our supply. The Biden administration has agreed to pay $5 billion for 10 million treatments. I previously thought this was 10 million pills as did the source, but I’ve been informed it was 10 million treatments, which means I was about to be happy to pay ten times what we’re actually paying. Kind of neat.
They’re going to make a profit from saving people’s lives. Outrageous!
The bad news, of course, is that Paxlovid Remains Illegal.
The Paxlovid Paradox
Scott Alexander is the latest to point out that this is murderous madness1. As I observed, and was later quoted at Marginal Revolution:
The trial was stopped due to ‘ethical considerations’ for being too effective. You see, we live in a world in which:
It is illegal to give this drug to any patients, because it hasn’t been proven safe and effective.
It is illegal to continue a trial to study the drug, because it has been proven so safe and effective that it isn’t ethical to not give the drug to half the patients.
Who, if they weren’t in the study, couldn’t get the drug at all, because it is illegal due to not being proven safe and effective yet.
So now no one gets added to the trial so those who would have been definitely don’t get Paxlovid, and are several times more likely to die.
But our treatment of them is now ‘ethical.’
For the rest of time we will now hear about how it was only seven deaths and we can’t be sure Paxlovid works or how well it works, and I expect to spend hours arguing over exactly how much it works.
For the rest of time people will argue the study wasn’t big enough so we don’t know the Paxlovid is safe.
Those arguments will then be used both by people arguing to not take Paxlovid, and people who want to require other interventions because of these concerns.
FDA Delenda Est.
Scott Alexander notes that the current prediction market median approval date is January 1, 2022, with a 92% chance of approval by March. I’ve asked Polymarket to put up a market I would trust somewhat more, which should be up in a few days. My guess is that this the current median estimate is somewhat too pessimistic.
He goes through and quickly destroys many plausible reasons why this delay would make sense. In the comments several attempts were made to raise additional objections, with concerns about the ‘manufacturing process’ being the closest to kind of maybe being able to pretend to be a real consideration (as opposed to the others, which were at best pretending to pretend). I still don’t think it does an especially good job of pretending to be a big enough concern to matter, and was disappointed by Scott’s respect for the objection, which I’ll discuss more in detail below. Whatever pills we have now were somehow manufactured, without expected delays we would have made a lot more of them a lot faster, and we could have dealt with that issue while doing the trial if it was the true bottleneck and the incentives had been reasonable.
This is noteworthy, not because I think it’s identifying the causal mechanism (I think it mostly isn’t) but because it describes a world that makes more sense and stopping the trial unethical but a reasonable response to incentives. If only this were about people who responded so well to pure dollar incentives our problems would be so much easier to fix.
The Body Count
How bad is this delay? How many people are going to needlessly die?
Will is right. This is not hyperbole. It sure looks like a lot of people are going to needlessly die.
This is a system of anti-ethics, the reverse of actual ethics, because supporting rules in opposition to ethics shows your loyalties won’t be threatened by concerns about actual ethics.2
It is not hyperbole, but it is imprecise. What’s our best guess of how many Americans will die as a result of the FDA’s inaction on Paxlovid?
For a given person, Paxlovid saves their life if and only if:
They get Covid-19 and would have died without Paxlovid.
Covid-19 gets detected early enough to use Paxlovid.
They are given Paxlovid.
They live.
We would need to subtract any opportunity costs or side effects from Paxlovid, but for now it seems reasonable to assume the side effects are small when used on Covid-19 positive patients when compared to the benefits, and that the opportunity costs are also small since Paxlovid should combine well with other known-to-be-effective options.
We need to estimate the number of people who are going to die between now and approval in order to do step one. My guess is that 50,000 is somewhat high, since deaths lag and the surge is only starting now, so if we are starting now I’d use more like 40,000. But it’s better to start at least two weeks ago if we’re computing the overall death toll, so it’s more like 60,000 for those purposes. For now, we’re going to assume that ‘as soon as the clinical trial data is clear as day plus enough time to call and have a meeting’ is a reasonable starting point.
Step four only works about 90% of the time, so we need to multiply the number of dead people by 90%, as the other 10% couldn’t have been saved. That’s the other easy one.
Then we need to figure out how many of those who die detect Covid-19 fast enough to get Paxlovid when it would be effective. That’s trickier, especially with our current lack of tests, although that’s also the FDA’s fault. The failure rate that matters is that among cases that would be fatal without Paxlovid, so there’s biases pointing both ways—cases that are are harder to detect are mostly harmless, but cases where the patient doesn’t seek treatment fast enough tend to be deadlier. Given that our best guess is that most cases are never detected, the first consideration dominates. Also the existence of Paxlovid would make people more inclined to get tested. What is our best guess as to the Paxlovid-world’s detection rate? I’m going to guess this is something like 90% as well, the window is reasonably wide, but I’d also believe substantially lower, so I’ll use 80% to cover model uncertainty.
That leaves the big question. If the FDA approved Paxlovid, how many doses would be available? Would they be distributed to the people who needed them? Would those doses represent doses that would not otherwise be used, or would they move consumption forward that would have happened anyway?
This is trickier, and I don’t have the answer if we look only using causal decision theory on this particular choice. Paxlovid is 90% effective but the effectiveness per pill is many orders of magnitude lower since most people would recover anyway, and there were only 7 deaths in the entire clinical trial control group despite them enrolling over 1,000 patients.
It’s entirely possible that, while making Paxlovid widely available today instead of widely available on January 1 would save about 30,000 lives, given the delays already built into the system, the delay on approval won’t on net cost many lives. If it’s true that Pfizer will have about 180,000 treatments by end of year (Tweet was corrected), and each patient takes ten pills, then we could only treat 180,000 patients, and that probably saves about 5,000 lives (depends on how selective you think we can be in finding a vulnerable group, we can do better than the study but earlier my estimate of this was clearly too high and I was thinking 20k), less if the bottleneck would have continued for a while, but given how close it is the bottleneck presumably wouldn’t last too much longer. Which is only a moderate decrease.
This is relevant to the value of doing an end run of sorts around this particular decision point, or leaning on the FDA regarding it, if you don’t count impacts on future other decisions.
Responding to Incentives
That still treats the manufacturing process as if it couldn’t have gone any faster, or at least that the decisions made in how fast to go weren’t interlinked with expected future FDA behavior, at least in the past. If Pfizer would have expected to FDA to approve Paxlovid a month sooner, that’s a strong reason to get more pills made faster, even when there isn’t yet full confidence that the drug is going to work. Similarly, an advance commitment to paying a higher price for the first however many pills delivered quickly, conditional on the drug working, and ideally to also cover some costs if it failed, would have done a similar thing.
It’s a magician’s trick. The FDA causes the delays, and not in a strange functional decision theory or acausal trade kind of way. They do it in the direct obvious kind of way. It’s worth quoting the rest of Eliezer’s thread.
If one is counting the invisible graveyard now that it is about to both grow far larger and be much more highly visible, one needs to at least count anyone denied existing known-to-be-safe-and-effective life-saving medicine, and those for whom denial is the result of expected delays and compensation schemes preventing timely mass production. The question is the extent to which one counts those for whom treatments were never developed, or were only developed after they already died, because of the expectation of FDA inaction.
Thus, my answers are:
For an unreasonable narrow interpretation that only counts those for whom the medicine was already sitting in a warehouse waiting for approval, and treat that shortage as a ‘whoops, making things is hard and takes time’ rather than a directly caused effect, the FDA is going to directly murder about 5,000 people in the United States.
For a compromise position where we count those who died while the FDA was holding up approval that would have been saved, the FDA is going to murder about 30,000 people in the United States if you start now, somewhat more if you start when we learned Paxlovid was safe and effective minus reasonable time to confirm this.
For a position that is comparing to the counterfactual of what would have happened if Pfizer knew in March 2020 that any pill it developed would be given out whenever there was sufficient Bayesian evidence it was safe and effective, and that they would be well-compensated for having more pills available quickly, the death toll is much higher, and especially much higher overseas.
As an exercise, one might want to consider the same calculations to delays in approval of the vaccines, remembering that Moderna and Pfizer both had their vaccine candidates within two days of starting work. And one could also do the same calculation on the continuing failure to approve and allow tests. Or one could include all the psychological and economic damage that could have been prevented, or the lives not fully lived, while we are forced to wait.
Footnotes (not working right yet, sorry):
This is not Scott’s language, nor is it to ascribe murderous madness to any particular person. It is still important to call things by their right names.
One may wish to consider that perhaps this system of anti-ethics is part of a broader system that is also anti-epistemic, anti-virtue and anti-value in general, and that there is what might be called The Implicit Conspiracy that implicitly cooperates with and rewards with power those who demonstrate their dedication to this reversed system, and punishes those who fail to demonstrate this dedication or reward those who do show it. Would the resulting world look like our world?
Note: I was moving quickly, and did the wrong calculation for the lower bound earlier. It is now 5k rather than 20k.
I disagree with this, in that my lowest count on FDA related deaths is approximately zero, or less than 100, for the exact reason that Gurkenglas mentioned below. The post already recognizes that there is a manufacturing bottleneck, so if we assume that the FDA approval process has not caused the bottleneck, the 180 000 pills available by the end of the year will be given out later, with an admittedly non zero loss of efficiency.
Why should we assume that the FDA approval has not caused that bottleneck? Because we should assume that if a prediction market knows the drug is getting approval, Pfizer knows it as well, and will manufacture as quickly as possible once they know the efficacy numbers. Sells of the drug will be supply, and not demand constrained, even through 2022. 50 million doses (the projected capacity for 2022) is not enough to cover the developed world.
Weak evidence: Molnupiravir was approved in the UK earlier this month, but is still not available. I am not certain whether that is entirely due UK’s decision to further test it in a trial for vaccinated people (which indeed will cost many lives).
Caveats: I am not saying that the FDA would act differently if there was no manufacturing bottleneck, just that in this case the slow FDA decision is much smaller.
Pfizer might also have some uncertainty because they do not know whether the FDA will approve the use of the drug for vaccinated people, which indeed might reduce upfront investment. But that is not directly related to the delay of the approval in itself.
It does happen a drug in a clinical trial performs great and the trial is stopped. What usually happens when that happy event occurs is the control group is moved over to the new drug. But keep in mind many drugs are being tested for chronic conditions like cancer, diabetes, or heart disease. The miracle drug doesn’t cure but just does a better job at holding the underlying disease at bay.
This is a bit different because it is a bit like Tamiflu. You use it within a few days when symptoms appear. When this trial was stopped, most likely there was no control group to move onto the drug. Those that didn’t get the drug went into their Covid infection and either recovered or not.
Manufacturing is not crazy at all. The drugs made for clinical trials are made in small batches which require their own control and checks to make sure the drug can be made without impurities and with the correct dose. It’s easy to forget but it really is amazing that you can go to a dollar store, buy a bottle of 350mg aspirin and every single pill in that bottle will have no more or no less than 350mg of active ingredient. The set up to make a few thousand pills for the purpose of a trial is not the same as the prime time type set up to churn those pills out by the millions.
The FDA approves both the drug and the company’s manufacturing operation. That begs the question does Pfizer have their manufacturing ready to go today, just waiting for FDA inspectors to say “turn it on”? I don’t know, there’s a lot of different drugs being tested and I doubt many companies can afford to keep a factory tooled and set to make the pill but otherwise idle the moment a good result comes in from the trials. Most likely the factories are making other pills, many probably more important than simple aspirin, and it is not a simple matter of just flipping a switch and changing over to this pill. I note this trial was completed earlier than expected so possibly Pfizer had slotted manufacturing time later on, in expectation the trial would be ongoing right now. So this is also fairly a question of when can Pfizer bring their manufacturing online and I’m sure they are more than eager to do so.
Wondering if you could explain what I’m missing about the “it’s too effective” argument, as I see you making it every weak, and I keep scratching my head. The idea that the testing was halted for ethical reasons for the drug being “too effective” seems appropriate to me for the reason that keeping the trial running, which means denying certain participants treatment, and thus them dying, is, in my mind, unethical. Whether or not that means the drug has the chance to make it through the red tape of legality proceedings efficiently seems like a completely separate issue. So, shouldn’t the argument be directed more toward why seemingly effective drugs take so long to be approved, and not on how moronic it is to stop trials when things see so lopsided? Mind helping me understand the point you’ve been making? I just keep stumbling over it on the weekly, and I want to move on! Thanks. :)
If the drug hasn’t been approved, then those participants weren’t going to get it anyway, so they’re no worse off in the trial.
I think the point is, they could have had other interventions had they not been in the trial. Which is why I believe they stopped it. Those that got placebo were stuck having to be denied other forms of intervention so as not to corrupt the study. (as I understand it)
What do you think of this new South African variant that seems to be massively out-competing Delta?
https://twitter.com/jburnmurdoch/status/1463956686075580421
Just read about that. I think we need more information before updating our overall assessments?
Why are you quoting without correction someone who thinks 5 billion divided by 10 million is 500,000 (it’s 500)?
Um, I was assuming everyone would understand that this wasn’t correct? Are we so fargone I can’t do this?
Clicking on the footnote link doesn’t automatically scroll to the footnotes (when clicked on LW, I haven’t checked other places (yet)).
Only opportunity costs that affect the number of lives, per the measure:
I asked mods (well, emailed Oliver about) how to make the footnotes work right—they’re good on Substack but broken in WP and LW then imports from WP. Growing pains of switching to the SS editor, whereas the WP editor didn’t have footnotes worth using at all.
Do you have references for the Paxlovid paradox? How transparent is the FDA about what their reasoning is, and what they are doing during the approval process?
As is, some people are asking about specifics, that could make the slow approval look more reasonable. For example:
Did they actually stop the Paxlovid trial, or did they transform it from a “is this drug effective?” to “is this drug safe?” trial by giving the control group Paxlovid and continuing to monitor everyone?
Are they currently looking at manufacturing techniques, labeling, dosing, and such? Or are they just waiting until March to give a generic thumbs-up?
Overall, I’m wondering how strong a case could be made to normal people for whom “but safety!” is a strong counterargument to “30,000 deaths!”.
Do you not need a control group in a safety trial?
The most unreasonably narrow interpretation would say that since production lags behind patients in need of treatment anyway, the FDA murders none because every pill not used today is an additional pill used later.
Every pill used today is not a pill not used later. More pills will be made later, for as long as there is a need for them.
Pretty sure I directly addressed this, as did Eliezer in his thread (and there’s at least one more explanation I didn’t mention, which is that you can target the vulnerable over time instead of giving it to everyone if you’re short). Also pretty sure this isn’t true even if you accept the premise.
The direct address being earlier approval causing more production ramp-up, I’m guessing. The point in parentheses is a good one, though the Other could reply that the resulting murders are a lot fewer when you assume triage in both scenarios. If we can, say, treat a million people out of 10 million cases, 8 million of which are in the next year, then with late approval we can only treat the most at-risk of the latter 8 million, while with early approval we can treat the most at-risk of 10 million. That’s 200000 treatments used on the 90-100 percentile risk bracket instead of the 87.5-90 percentile risk bracket.
Is there some way I can donate Bitcoin to get a Mexican meth lab to shift their production to Paxlovid? Or any other action that helps get Paxlovid to people, that doesn’t require being in the pharmaceutical or healthcare industries and doesn’t assume that complying with whatever the FDA says is required?
I continue to be disappointed by people’s compliance with authority during this pandemic. The perceived dangers of noncompliance seem almost entirely imaginary to me.
In Germany you have our government stopping vaccination they believe to be illegal (https://www.ndr.de/nachrichten/schleswig-holstein/Nicht-zugelassener-Impfstoff-Behoerden-stoppen-Impfaktion-in-Luebeck,impfaktion178.html) even when it’s not that clear that any law got violated.
You’re right; my statement was too broad, and there are definitely types of noncompliance whose dangers are very real. I should have said that there seem to be cases in which the dangers of noncompliance are almost entirely imaginary, and people don’t seem to be bothering to check; they notice that official policy has predictably terrible outcomes, and complain about it, but stop short of seriously considering what would happen if they just didn’t follow the policy.
The CDC’s botched test kits are the first and best example I know of. It seems to me that if the kit recipients had just used the good components they already had lying around instead of the CDC’s bad ones, and then were asked to justify what they had done, their response would have been no more blameworthy, and led to little more consequence, than “well, this kit I bought at Ikea had a busted screw, so I just used this other screw with the same threads and length that I had lying around”. That this didn’t happen makes me think that compliance for compliance’ sake was the recipients’ main motivation. I find that disappointing.