A vaccine brought to clinical trials has already overcome many more hurdles than this has.
I’m not sure what your level of background knowledge is, but I heard that the Moderna vaccine was designed in two days. Clearly they did not do any significant in vitro or in vivo testing in that timespan. Maybe they did some in-vitro before human trials, I don’t know; that would support an argument against using “vaccine brought to clinical trials” as a reference class.
But the deeper point which this is trying to operationalize is “vaccine design just isn’t that hard”, in the sense that we don’t need to test many designs to find one which works. People basically-know-how-to-design-vaccines, maybe not to quite the same extent as people basically-know-how-to-design-bridges, but to enough of an extent that experimental verification just isn’t necessary in order to get a >50% chance that the design works, especially for relatively-mechanical designs like mRNA or peptides.
Under this view, the reasons we don’t see nearly-every vaccine trial succeed are (1) commercial vaccines are harder than lab (especially if you want no boosters, easy logistics, etc), and (2) diseases which are harder-than-average will naturally end up with disproportionately many trials, and (3) out-of-date companies take time to die off.
The in vitro testing had already been done before those two days; they had the basic structure of the vaccine known, so once they had a virus sample they could fill in the blank (the spike protein of this particular virus rather than another in its ‘family’) with high confidence that it would work. One of the two days, IIUC, was spent synthesizing a sample vaccine and running some very-short-term tests.
This, again, has only simulation to support that it’s hitting the correct target at all. There is no indication that any of that has been done, by the authors or anyone else; IIUC there is not a clear path for doing short-term tests for this type of vaccine.
Also, it’s not my background knowledge that you should be comparing to, it’s Sarah’s. And I literally believe there is no one in the world who can be more trusted to reason clearly, well-informedly, correctly, and with humility and arrogance in their respective correct places than Sarah Constantin. Evaluating biomedical research has been her job for many years, with some gaps, and she’s really good at rationality, Aumann-level good. The bare fact that Sarah C thinks this is very unlikely to work is conclusive on its own.
How does the “vaccine design just isn’t that hard” align with these points?
a) Average time to develop a vaccine for a new virus is many years
b) There is still no HIV vaccine after 35 years of well-funded research
c) Until a few months ago, there were no approved coronavirus vaccines for humans
I’m prepared to accept that “bureaucracy” is the main cause for the delays in standard big company vaccine development and approval.
But if it’s easy to develop vaccines, why has there been no coronavirus vaccine previously? Why is there still no vaccine for SARS 1 or MERS or the common cold? Why was this Radvac idea or something similar not rolled out pre-Covid? (or was it? maybe nasal vaccines are easier?)
Anyway, I’m just stuck on the logical conflict between “it’s easy to develop a coronavirus vaccine” and “we’ve never had one (approved) before.” Any thoughts?
First, I expect a disproportionate number of vaccine trials are for “unusually difficult” viruses, like HIV. After all, if it’s an “easy” virus to make a vaccine for, then the first or second trial should work. It’s only the “hard” viruses which require a large number of trials.
But if it’s easy to develop vaccines, why has there been no coronavirus vaccine previously? Why is there still no vaccine for SARS 1 or MERS or the common cold? Why was this Radvac idea or something similar not rolled out pre-Covid?
I expect this is still mainly a result of regulatory hurdles. Clinical trials are slow and expensive, so there has to be a pretty big pot of gold at the end of the rainbow to make it happen. Also, companies tend to do what they already know how to do, so newer methods like mRNA or peptide vaccines usually require a big shock (like COVID) in order to see rapid adoption.
I agree with the point of your comment, that vaccines brought to clinical trials is a suboptimal reference class. However, I think that this is a locally invalid argument:
Would you conclude that, because some lines of code can navigate a rocket to the moon, that your code is pretty likely to navigate a rocket to Mars?
A computational model plus grounding in theory, if done right, should increase our confidence in the the efficacy of a sequence of peptides taken from the virus above the efficacy we’d assume for a random sequence of peptides.
How much? Can’t say.
As others have pointed out here, we on the other hand are comparing a new and perhaps much more effective means of designing a vaccine to the methods that were used from 2000-2015, which may be less effective. Hence, perhaps the reference class is suboptimal in the opposite direction as well.
I have no way to know how to weigh these competing factors. So I think the best thing to do is to start with the basic formula I concocted above, then modify it based on our intuitions about these other factors.
Alternatively, you could very justifiably stick with the rule “I don’t take untested medications.” Although as someone else pointed out, if you have that rule then perhaps you should also make sure to not use any drugs? I don’t have the answer, but wanted to try and provide some clarity for people who are considering breaking the “take no untested medications” rule.
This is an inappropriate reference class. This has no in vitro testing conducted; it’s entirely a computational model. ““Peptide” just means “sequence of amino acids.” Would you conclude that, because some lines of code can navigate a rocket to the moon, that your code is pretty likely to navigate a rocket to Mars?”. A vaccine brought to clinical trials has already overcome many more hurdles than this has. Generally in vitro testing (I think both for safety and efficacy), in vivo safety testing (in rats!), and some scaled-up testing in other animal models.
This isn’t a vaccine candidate. This is a promising research lead for a vaccine candidate.
I’m not sure what your level of background knowledge is, but I heard that the Moderna vaccine was designed in two days. Clearly they did not do any significant in vitro or in vivo testing in that timespan. Maybe they did some in-vitro before human trials, I don’t know; that would support an argument against using “vaccine brought to clinical trials” as a reference class.
But the deeper point which this is trying to operationalize is “vaccine design just isn’t that hard”, in the sense that we don’t need to test many designs to find one which works. People basically-know-how-to-design-vaccines, maybe not to quite the same extent as people basically-know-how-to-design-bridges, but to enough of an extent that experimental verification just isn’t necessary in order to get a >50% chance that the design works, especially for relatively-mechanical designs like mRNA or peptides.
Under this view, the reasons we don’t see nearly-every vaccine trial succeed are (1) commercial vaccines are harder than lab (especially if you want no boosters, easy logistics, etc), and (2) diseases which are harder-than-average will naturally end up with disproportionately many trials, and (3) out-of-date companies take time to die off.
The in vitro testing had already been done before those two days; they had the basic structure of the vaccine known, so once they had a virus sample they could fill in the blank (the spike protein of this particular virus rather than another in its ‘family’) with high confidence that it would work. One of the two days, IIUC, was spent synthesizing a sample vaccine and running some very-short-term tests.
This, again, has only simulation to support that it’s hitting the correct target at all. There is no indication that any of that has been done, by the authors or anyone else; IIUC there is not a clear path for doing short-term tests for this type of vaccine.
Also, it’s not my background knowledge that you should be comparing to, it’s Sarah’s. And I literally believe there is no one in the world who can be more trusted to reason clearly, well-informedly, correctly, and with humility and arrogance in their respective correct places than Sarah Constantin. Evaluating biomedical research has been her job for many years, with some gaps, and she’s really good at rationality, Aumann-level good. The bare fact that Sarah C thinks this is very unlikely to work is conclusive on its own.
How does the “vaccine design just isn’t that hard” align with these points?
a) Average time to develop a vaccine for a new virus is many years
b) There is still no HIV vaccine after 35 years of well-funded research
c) Until a few months ago, there were no approved coronavirus vaccines for humans
I’m prepared to accept that “bureaucracy” is the main cause for the delays in standard big company vaccine development and approval.
But if it’s easy to develop vaccines, why has there been no coronavirus vaccine previously? Why is there still no vaccine for SARS 1 or MERS or the common cold? Why was this Radvac idea or something similar not rolled out pre-Covid? (or was it? maybe nasal vaccines are easier?)
Anyway, I’m just stuck on the logical conflict between “it’s easy to develop a coronavirus vaccine” and “we’ve never had one (approved) before.” Any thoughts?
Good questions.
First, I expect a disproportionate number of vaccine trials are for “unusually difficult” viruses, like HIV. After all, if it’s an “easy” virus to make a vaccine for, then the first or second trial should work. It’s only the “hard” viruses which require a large number of trials.
I expect this is still mainly a result of regulatory hurdles. Clinical trials are slow and expensive, so there has to be a pretty big pot of gold at the end of the rainbow to make it happen. Also, companies tend to do what they already know how to do, so newer methods like mRNA or peptide vaccines usually require a big shock (like COVID) in order to see rapid adoption.
I agree with the point of your comment, that vaccines brought to clinical trials is a suboptimal reference class. However, I think that this is a locally invalid argument:
A computational model plus grounding in theory, if done right, should increase our confidence in the the efficacy of a sequence of peptides taken from the virus above the efficacy we’d assume for a random sequence of peptides.
How much? Can’t say.
As others have pointed out here, we on the other hand are comparing a new and perhaps much more effective means of designing a vaccine to the methods that were used from 2000-2015, which may be less effective. Hence, perhaps the reference class is suboptimal in the opposite direction as well.
I have no way to know how to weigh these competing factors. So I think the best thing to do is to start with the basic formula I concocted above, then modify it based on our intuitions about these other factors.
Alternatively, you could very justifiably stick with the rule “I don’t take untested medications.” Although as someone else pointed out, if you have that rule then perhaps you should also make sure to not use any drugs? I don’t have the answer, but wanted to try and provide some clarity for people who are considering breaking the “take no untested medications” rule.