Given that we can show governance was definitely a critical bottleneck (production proceeded as soon as governance allowed)
One possibility is that the FDA, already concerned about vaccine hesitancy, put off approval until they knew the manufacturers had scaled up adequately.
I think under your hypothesis, the timing was something like this:
Producers scale up and have lots of vaccine ready to go months prior to December (when Pfizer was approved). The FDA refuses to approve the vaccines for months, despite the producers being ready to go with a big stockpile and production capacity, until December. Once they give emergency authorization, the vaccine distribution begins immediately on the stockpile they’d built up.
An alternative hypothesis is this:
Producers can make enough vaccine for early trials, but don’t manage to scale up to produce millions of doses until December. The FDA takes the opportunity to gather data on safety and efficacy until that time. However, they don’t want to look like they’re issuing emergency approval just because the vaccine’s ready to go. They want it to look like it’s because they have reached satisfaction with the safety and efficacy data. So that’s the line they put out to the public. Yet in reality, they give approval right around the time when producers, distributors, and vaccination sites are ready to go.
It’s really not obvious to me that one or the other of these is correct. In the former, regulatory decision-making is the bottleneck. In the latter, regulators are striving to make regulation have the appearance of a bottleneck, when in fact the true bottleneck is manufacturing capacity.
why is the burden of proof on us to show that no other bottlenecks happened to also be critical.
Manufacturing a novel vaccine at mass scale, and setting up the cold-storage and clinic sites to distribute and inject it, was a huge logistical project. It has lots of technical details. The FDA knows how to evaluate safety and efficacy; it’s what they do. By contrast, the manufacturing was a novel challenge. It probably put strains on the global supply chain and the world’s vaccine manufacturing expertise. Overall, it’s a more complex and novel challenge than testing and regulation, with far more ways for delays to crop up.
So when there’s a reasonably plausible explanation for why governance and manufacturing bottlenecks appear to have coincided (with the root cause actually being manufacturing), and we also know that the manufacturing is the novel/complex part, and we can more obviously see how politicians and manufacturers would both stand to benefit from an efficient rather than an inefficient process, it seems like you should at least take that possibility very seriously?
we can more obviously see how politicians and manufacturers would both stand to benefit from an efficient rather than an inefficient process
In a world where it would have been possible to vaccinate every person on earth for the cost of $1000 because we have very efficient technology for vaccination nobody would have paid billions for vaccines and those companies wouldn’t have made a lot of profit.
Moderna and Pfizer used the most complex technology for producing vaccines and are also paid more per vaccine then other companies (and those contracts were made before we knew anything about effectiveness).
Using simply solutions is just not the way profits are made in biopharma.
Producers can make enough vaccine for early trials, but don’t manage to scale up to produce millions of doses until December. The FDA takes the opportunity to gather data on safety and efficacy until that time. However, they don’t want to look like they’re issuing emergency approval just because the vaccine’s ready to go. They want it to look like it’s because they have reached satisfaction with the safety and efficacy data. So that’s the line they put out to the public. Yet in reality, they give approval right around the time when producers, distributors, and vaccination sites are ready to go.
If that was correct we’d expect that FDA approval process tends to vary by how quickly manufacturing capacity can be ramped up. The key criticism I have of the FDA is not allowing challenge trials. If you are correct and that was intentional to give time for capacity to ramp up, we would expect to see drugs which can be ramped up more quickly use challenge trials—but that’s not something we’ve seen. If anything this was a ridiculously fast approval, for a process that usually takes years.
Furthermore I find it highly unlikely that producers would ramp up production just as quickly before results of trials are in as afterwards. So by not allowing challenge trials, less was invested in ramping up, so the ramp up itself (if it was a critical bottleneck) would have taken longer.
and we can more obviously see how politicians and manufacturers would both stand to benefit from an efficient rather than an inefficient process
The overwhelming evidence I’ve ever seen is that politicians and government orgs are highly inneficient. My prior on them being efficient here is extremely low.
If that was correct we’d expect that FDA approval process tends to vary by how quickly manufacturing capacity can be ramped up.
I think the right reference class here is emergency use authorizations, not FDA approval. Under my model, yes, their emergency use authorization timeline would vary by the speed of the manufacturing ramp-up. I might expect a global pandemic context to be a special context even among EUAs, but let’s not add epicycles at this point.
Another factor to consider is that the interplay of manufacturing and regulation may vary depending on whether the treatment is among the first, or whether it is a latecomer. For example, Novavax only got preliminary data on its vaccine in Jan 2021. It’s only started scaling up after that. When they got the prelim data, the CEO said it would take 5-6 months to scale up to 150 million doses.
So why is it that they can get to that scale in 5-6 months, if it took 8 months+ to scale up vaccine production for Pfizer or Lonza, as I claim? One possibility is that their vaccine is easier to manufacture than others. Another is that they’re able to use global production capacity that’s been scaled up over the last year and a half.
A third, of course, is that it really doesn’t take more than 5-6 months to scale up to 150 million+ quality-controlled doses if you’re adequately incentivized. But then, we’d expect Pfizer to be cranking out that much right now. The number is closer to 20 million per month, even though they’re a much bigger company than Novavax.
Furthermore I find it highly unlikely that producers would ramp up production just as quickly before results of trials are in as afterwards.
Novavax’s decision to start scaling up only after getting their prelim results is some evidence in favor of your argument. Likewise, the fact that, back in July 2020, Operation Warp Speed offered them $1.6 billion to scale up, but only after their data was in showing that the vaccine works.
It makes perfect sense that companies are risk-averse, and don’t want to invest lots of money in scaling up without good data showing it’s not going to be a wasted investment. An HCT would give them that confidence faster.
The overwhelming evidence I’ve ever seen is that politicians and government orgs are highly inneficient. My prior on them being efficient here is extremely low.
The important comparison here is the inefficiency of the politicians vs. the difficulty of the manufacturing process. It doesn’t matter how slow the politicians go if the manufacturing scale-up process is even slower than that.
Anyway, I find that overall, there’s reason to think and some evidence in favor of regulation being an important source of slowdown. I find your argument that HCTs could have reduced uncertainty around trial outcomes, thus accelerating scale-up efforts, particularly compelling.
At the same time, the discrepancy between Pfizer and Novavax’s current and projected scale-up timelines, even months post-EUA for Pfizer, makes me think that manufacturing must also be an important source of slowdown. That may be a tighter constraint than improving regulation. We can’t kick science and manufacturing to make it go faster, but sometimes we can kick the government to make it go.
From a political perspective, then, it makes some sense to focus on the failures of government. From an epistemic standpoint, though, I think we really do need to spend more time understanding the manufacturing problems.
One possibility is that the FDA, already concerned about vaccine hesitancy, put off approval until they knew the manufacturers had scaled up adequately.
I think under your hypothesis, the timing was something like this:
Producers scale up and have lots of vaccine ready to go months prior to December (when Pfizer was approved). The FDA refuses to approve the vaccines for months, despite the producers being ready to go with a big stockpile and production capacity, until December. Once they give emergency authorization, the vaccine distribution begins immediately on the stockpile they’d built up.
An alternative hypothesis is this:
Producers can make enough vaccine for early trials, but don’t manage to scale up to produce millions of doses until December. The FDA takes the opportunity to gather data on safety and efficacy until that time. However, they don’t want to look like they’re issuing emergency approval just because the vaccine’s ready to go. They want it to look like it’s because they have reached satisfaction with the safety and efficacy data. So that’s the line they put out to the public. Yet in reality, they give approval right around the time when producers, distributors, and vaccination sites are ready to go.
It’s really not obvious to me that one or the other of these is correct. In the former, regulatory decision-making is the bottleneck. In the latter, regulators are striving to make regulation have the appearance of a bottleneck, when in fact the true bottleneck is manufacturing capacity.
Manufacturing a novel vaccine at mass scale, and setting up the cold-storage and clinic sites to distribute and inject it, was a huge logistical project. It has lots of technical details. The FDA knows how to evaluate safety and efficacy; it’s what they do. By contrast, the manufacturing was a novel challenge. It probably put strains on the global supply chain and the world’s vaccine manufacturing expertise. Overall, it’s a more complex and novel challenge than testing and regulation, with far more ways for delays to crop up.
So when there’s a reasonably plausible explanation for why governance and manufacturing bottlenecks appear to have coincided (with the root cause actually being manufacturing), and we also know that the manufacturing is the novel/complex part, and we can more obviously see how politicians and manufacturers would both stand to benefit from an efficient rather than an inefficient process, it seems like you should at least take that possibility very seriously?
In a world where it would have been possible to vaccinate every person on earth for the cost of $1000 because we have very efficient technology for vaccination nobody would have paid billions for vaccines and those companies wouldn’t have made a lot of profit.
Moderna and Pfizer used the most complex technology for producing vaccines and are also paid more per vaccine then other companies (and those contracts were made before we knew anything about effectiveness).
Using simply solutions is just not the way profits are made in biopharma.
If that was correct we’d expect that FDA approval process tends to vary by how quickly manufacturing capacity can be ramped up. The key criticism I have of the FDA is not allowing challenge trials. If you are correct and that was intentional to give time for capacity to ramp up, we would expect to see drugs which can be ramped up more quickly use challenge trials—but that’s not something we’ve seen. If anything this was a ridiculously fast approval, for a process that usually takes years.
Furthermore I find it highly unlikely that producers would ramp up production just as quickly before results of trials are in as afterwards. So by not allowing challenge trials, less was invested in ramping up, so the ramp up itself (if it was a critical bottleneck) would have taken longer.
The overwhelming evidence I’ve ever seen is that politicians and government orgs are highly inneficient. My prior on them being efficient here is extremely low.
I think the right reference class here is emergency use authorizations, not FDA approval. Under my model, yes, their emergency use authorization timeline would vary by the speed of the manufacturing ramp-up. I might expect a global pandemic context to be a special context even among EUAs, but let’s not add epicycles at this point.
Another factor to consider is that the interplay of manufacturing and regulation may vary depending on whether the treatment is among the first, or whether it is a latecomer. For example, Novavax only got preliminary data on its vaccine in Jan 2021. It’s only started scaling up after that. When they got the prelim data, the CEO said it would take 5-6 months to scale up to 150 million doses.
So why is it that they can get to that scale in 5-6 months, if it took 8 months+ to scale up vaccine production for Pfizer or Lonza, as I claim? One possibility is that their vaccine is easier to manufacture than others. Another is that they’re able to use global production capacity that’s been scaled up over the last year and a half.
A third, of course, is that it really doesn’t take more than 5-6 months to scale up to 150 million+ quality-controlled doses if you’re adequately incentivized. But then, we’d expect Pfizer to be cranking out that much right now. The number is closer to 20 million per month, even though they’re a much bigger company than Novavax.
Novavax’s decision to start scaling up only after getting their prelim results is some evidence in favor of your argument. Likewise, the fact that, back in July 2020, Operation Warp Speed offered them $1.6 billion to scale up, but only after their data was in showing that the vaccine works.
It makes perfect sense that companies are risk-averse, and don’t want to invest lots of money in scaling up without good data showing it’s not going to be a wasted investment. An HCT would give them that confidence faster.
The important comparison here is the inefficiency of the politicians vs. the difficulty of the manufacturing process. It doesn’t matter how slow the politicians go if the manufacturing scale-up process is even slower than that.
Anyway, I find that overall, there’s reason to think and some evidence in favor of regulation being an important source of slowdown. I find your argument that HCTs could have reduced uncertainty around trial outcomes, thus accelerating scale-up efforts, particularly compelling.
At the same time, the discrepancy between Pfizer and Novavax’s current and projected scale-up timelines, even months post-EUA for Pfizer, makes me think that manufacturing must also be an important source of slowdown. That may be a tighter constraint than improving regulation. We can’t kick science and manufacturing to make it go faster, but sometimes we can kick the government to make it go.
From a political perspective, then, it makes some sense to focus on the failures of government. From an epistemic standpoint, though, I think we really do need to spend more time understanding the manufacturing problems.