“MY WHY” (for my admittedly simply model)… Is… Basically… Uh… a thing I experience often is that I have a sort of “models inside of models” expectation that seems rarely to be applied in practice even by so-called experts?
For example during Feb/Mar 2020 there were people talking about how “X% of patients are asymptomatic” but they were making these assertions based on single snapshots of infection cohorts in whom the epidemic was actively moving. So some of the people (like on a cruise ship) might have been PRE-symptomatic (because maybe checking for symptoms 14 days later would turn something up) rather than A-symptomatic (as a general property of their entire disease course). People were using the terminology willy nilly, and no one was tracking any of it precisely… almost no one was thinking of it like they were getting momentary glimpses of a sort of state machine or advertising funnel or something, where each person might get a slightly different ride along a slightly different path AND ALSO might be at a different step along whatever ride they will end up having taken.
Another very common failing is that people think that a group mean implies a group median. If there are bimodal responses to something, and then a group summary is given… is the group’s denominator over “all treated” or “all who were treated and had a followup confirming an adequate response”?
BOTH of these errors have a common cause of “assuming homogeneous efficacy and assuming competent followup at the clinical level” and in my experience neither of these assumptions are strongly justified. They constantly fail, and people are constantly acting surprised about it.
Failures of followup are ALSO why we couldn’t get people properly quarantined at the beginning of this disaster.
Often patients DO NOT WANT to have “the system” FOLLOW them.
The study you linked to seems to have somewhat solved “the standard problems with the ‘lost to followup’ state” that is the bane of so many time course studies. The design certainly seems to take the followup process very much into account (and I couldn’t find drop out rates from skimming or ^f and so maybe literally no one dropped out):
Something I’d like to call attention to here… in the paper you link the Extended Track had Bleed1 data from the extended cohort group, where they got (in some sense) to see how many people seroconverted from just one dose by week 5 or so...
Within the extended interval cohort, antibodies were detectable in 91% (62/68) at the first timepoint, at 5-6 weeks after the first vaccine, but this rose to 100% 2-3 weeks after the vaccine boost.
Recall that up in the abstract the paper summarizes the key result thusly:
Results: In donors without evidence of previous infection the peak antibody response was 3.5-fold higher in donors who had undergone delayed interval vaccination.
Suppose, hypothetically, that instead of 91% of people having “a seroconverting response” on the first shot it was only 28% of them?
(This would be almost understandable. The youngest person in that study was 80 years old! The whole study is on a group whose immune systems should be assumed to be decrepit and fragile from the raw fact of great age.)
Then if the second exposure brought this up to 100% seroconversion “somehow”, and the seroconverted “antibody levels” were gaussian (log normal?) among the seroconverted and 0 among the rest...
...Then that bimodal response could directly and cleanly justify claiming “antibody response was 3.5-fold higher” in some very fuzzy and general way (because 28% x 3.5 = 98%)
MY NORMAL EXPECTATION is for people to communicate in a fuzzy and general way :-(
The graph you included as a supporting claim was, I think, just the B panel from the totality of Figure 2 which is nice in many ways. Color coded! The horizontal axes are mostly aligned! Nice!
Note that in Panel A the two timepoints give basically the same levels of antibody response, with maybe some hint of a slow decline, but also overlap with Panel B’s separated ranges. Some in Panel A went up?? Weird. Probably stuff goes down (and sometimes up?) over time, in general?
The data in Panel A therefore seems consistent to me that “eventually” there is some roughly normal and acceptable level of “vaccinated at all, in an essentially bimodal way” that two doses reaches faster than typical?
This is what the two dose shot is designed to do in my mind: get ALMOST ALL of the patients (because of herd immunity benefits) to the state of “CLEANLY SEROCONVERTED” with the LEAST amount of measurement and need for followup (because followup is really hard).
Bleed2 of the standard group is “10 weeks post standard dose2”. There is no Bleed3 for either group out all the way at week 21. That third data collection event would be “10 weeks after dose2 for the extended group” and thus sorta comparable to the standard group’s Bleed1?
My hunch is that extended Bleed3 would show a decline from the extended Bleed2 measurement…
...maybe this prediction is a crux?
I could also imagine those slow risers in the standard group would STILL be going up by week 21?
Basically, I suspect that antibody levels eventually go down EVENTUALLY (over months and years), but also have some “sensitivity to dynamics over a timecourse” (which is probably not showing up here, not because it didn’t happen, but because it wasn’t measured).
I don’t know. My error bars are wide.
...
Also it would have been great to measure antibody levels for everyone on week 3, as Bleed0? More of the dynamics would be visible I think, and it would help characterize (and separate?) various members of the standard group in terms of seroconversion status before the second dose?
...
Basically, immunological science is more of an art than a science. It has a gazillion moving parts created under extreme adversity. Also, humans underestimate the difficulties of just doing the thing over and over.
People don’t get that “seroconversion” IS A THING. Sometimes it just doesn’t happen. That’s often the most important practical fact. Many bad vaccine designs end up with a “vaccine” whose seroconversion rate is non-zero but so low as to be impractical. This meta-analysis for measles (trying to find a relationship between age and seroconversion) shows numerous things that were tried in the clinic that had less than 50% rates for some kids.
The Pfizer/Moderna/mRNA design is weirdly effective from my perspective here. To take a second dose of a weirdly effective thing… uh… I guess? Sure. If empirically that works for this disease maybe it could or would help somehow in a way that eventually could be made sense of.
So if I have to pick ONE THING to assume about vaccine status at the INDIVIDUAL level, it will be “seroconverted or not” and then after knowing the answer to that, I assume “the rest will be very very complicated”.
I freely admit this is a simplified model. I just also think that any specific additional mechanistic second step to the modeling effort is likely to explode the combinatorial space of patient states to worry about, and yet also be a TINY epicycle, and the first of MANY epicycles.
The thing I’m asking for is: what’s the best second epicycle to add? What is the mechanism? If someone is already seroconverted, what would you measure to detect “that their mechanistic biological state is not ALREADY in the configuration that you’d be hoping to cause to improve via the administration of a third dose”?
And in the meantime: DELTA MIGHT BE MUTATING AROUND THE CURRENT VACCINE DESIGN and so a new design aimed at the new epitopes just obviously seems like it would address the main uncertainty in a central way.
RadVac for Delta is something I might pay for, and maybe something I might want to try to make on my own? Searching a bit: it looks like mixed third dose trials are already starting, so… <3!
A third dose that’s “the same as the first two” doesn’t interest me. The second one doing what it does seems to be empirically real, but I don’t know why the hell it empirically gives the results it does.
On your “WHY”, you seem to be presenting reasons why other people not believing your model shouldn’t count as strong evidence against it. Which is all fair. But I’m still curious for positive evidence to believe your model in the first place. Maybe this would be obvious if I knew more biology, but as it is, I don’t know why I should place higher credence in your model than any other model (e.g. the one at the bottom of this comment, if that counts).
...Then that bimodal response could directly and cleanly justify claiming “antibody response was 3.5-fold higher” in some very fuzzy and general way (because 28% x 3.5 = 98%)
As far as I can tell, “antibody response was 3.5-fold higher” just means that, on average, people in the extended dosing schedule had 3.5x more antibodies. I can’t tell whether you interpret it in some other way, or if you think this is a misleading way to describe things, or if you’re making some other point...?
The graph you included as a supporting claim was, I think, just the B panel from the totality of Figure 2 which is nice in many ways.
Yup!
The data in Panel A therefore seems consistent to me that “eventually” there is some roughly normal and acceptable level of “vaccinated at all, in an essentially bimodal way” that two doses reaches faster than typical?
Ok now I’m confused.
Do you think that all people on these graphs have reached a “normal and acceptable level of ‘vaccinated at all, in an essentially bimodal way’ ”?
If so, do you not think that there’s any important immunity difference between a single-vaccinated person around 1-10 on the graph, or a doubly-vaccinated person around 1000-10000?
Or if you think that only some of the people on this graph are immune, where do you think the line between immune and not-immune should be drawn on these graphs? (The distribution seems to be fairly continuous everywhere, to me, so it seems arbitrary to draw the line anywhere.)
Or if you think the important immunity difference isn’t captured by antibody-levels, what is it about?
And re “that two doses reaches faster than typical”; are you implying that the single-dosed people’s antibody response would’ve kept increasing beyond the 5-6 week mark and eventually gotten as high as the doubly-vaccinated people? That seems unlikely to me. (Other than maybe the few people where their antibodies did increase, but I’m happy to ignore them until I understand the most normal response curve better.)
My hunch is that extended Bleed3 would show a decline from the extended Bleed2 measurement…
Agreed.
The thing I’m asking for is: what’s the best second epicycle to add? What is the mechanism? If someone is already seroconverted, what would you measure to detect “that their mechanistic biological state is not ALREADY in the configuration that you’d be hoping to cause to improve via the administration of a third dose”?
Here’s one suggestion:
1. The more antibodies you have, the less probability of getting sick, the less probability of getting severe disease, etc.
2. More vaccines increases the number of antibodies you have.
3. Therefore you want to have more vaccines.
I would’ve thought (1) to be fairly uncontroversial? And the linked study seems to provide good evidence for (2) when going from 1 to 2 doses, increasing antibodies by roughly a factor of 100. And of course adding more vaccines will eventually stop adding more antibodies. But right now I don’t have any reason to believe in a big difference between going from 1->2 vaccines vs going from 2->3 vaccines (other than 2 vaccines being the general standard). So I wouldn’t be surprised if taking a 3rd vaccine could increase your antibodies by another order of magnitude.
Maybe you think this doesn’t provide enough of a “mechanism”? Biology being complicated, I’m very happy to take empirical data for what it is, and make extrapolations even if I don’t know what the mechanism is. Personally, I also don’t feel like I have any more mechanism for “vaccine have a fixed probability of causing antibodies if you don’t already have them, otherwise they don’t do much” than “vaccine typically increases antibodies by a lot regardless of whether you have them or not”. So when the evidence clearly indicates the latter, I will definitely believe it.
And yeah, also, if someone has the option, I agree that it seems probably better to get a different vaccine than the same vaccine again!
I sort of tapped out because “very long posts with an explosion of quotes” is a smell for me, but I wanted to continue because other indicators suggest “teaching and/or learning in good faith” <3
Finally posting now because of a big update from elsewhere...
On your “WHY”, you seem to be presenting reasons why other people not believing your model shouldn’t count as strong evidence against it. Which is all fair. But I’m still curious for positive evidence to believe your model in the first place.
For me, evidence happens at the point of measurement. Then often measurements are summarized in language by people who don’t think clearly, or worry about standard misinterpretations of simple measurements… so careful reading is sometimes required just to acquire evidence able to distinguish between models.
So for me, the default is to need to think about mechanistic timecourse evidence through the screen of “how it was confusingly explained to me” by people who often aren’t worried about mechanistic timecourse dynamics.
I kinda don’t care if people don’t believe my model, I just want my models to get better over time… and I’m happy to explain them to people, and I like teaching… but if people don’t believe me, then it is their tragedy that they believe false things, not my tragedy. (Conversely, people teaching me things is awesome!)
But to make my models better I don’t just import other people’s posterior believes about how a mechanistic system works, but rather see if my own model can “round trip” through my best guess of the raw data that they observed in a specific situation. If people have bad reasoning, then their posteriors are even less safe to import than otherwise...
FWIW, just tonight I got around to reading this cousin comment by Connor and it swiftly tipped me over almost entirely. Three doses… might work? Sure.
I already thought there were empirical reasons to think it, so for me I think the key words in Connor’s post started somewhere around:
And not only does it increase count, secondary responses vastly increase antibody affinity and produce different antibody types, e.g. the primary response is more IgM whereas secondary response produces more IgG and IgA (the latter aiding especially in mucosal immunity). [Citations for this can be found on pgs 413-414 of the Janeway immunobiology book, and I can maybe link pictures.]
The filter I have I think, is that I want to hear about mechanisms when it comes to biological theories.
I’m not saying button mashing doesn’t work. That plus “copy the winner” is how most actual technical innovation occurs and scales in practice most of the time. Its fine <3
But… a HUGE filter that avoids adding broken bits to my general reasoning capacities is whether someone can offer keywords that connects their proposed mechanism to ALL THE OTHER MECHANISMS in physics and chemistry and evolution and all of it.
I have paragraphs and paragraphs of text from my first attempt at a response, trying to explain “I don’t know and neither do you (but politely and at length)”.
They are deleted from this response. Maybe “two people debugging epistemics in the face of ignorance” is useful somehow for something, but I’m not attached to it. I could PM it maybe if you care?
Practical upshot: empirically more doses has worked, and now I have heard some “new magic mechanism words” from Connor, who seems to me to clearly knows his shit backwards and forwards and also seems to be in tentative favor of a third dose :-)
Maybe interesting: my main argument AGAINST a third dose is part of why I thought it might be smart to give single doses as fast as possible several months ago. Now that like… “mechanisms are mechanically different (giving more than just lots of IgM)” I feel like I learned enough to even notice errors in past thinking?
But also… weirdly(?) this same body of empirical results says that the second reaction works BETTER after … <missing mechanism that somehow is time dependent> has had 12 weeks to <do something> instead of just 3 weeks?
You found a neat paper! Thank you!
“MY WHY” (for my admittedly simply model)… Is… Basically… Uh… a thing I experience often is that I have a sort of “models inside of models” expectation that seems rarely to be applied in practice even by so-called experts?
For example during Feb/Mar 2020 there were people talking about how “X% of patients are asymptomatic” but they were making these assertions based on single snapshots of infection cohorts in whom the epidemic was actively moving. So some of the people (like on a cruise ship) might have been PRE-symptomatic (because maybe checking for symptoms 14 days later would turn something up) rather than A-symptomatic (as a general property of their entire disease course). People were using the terminology willy nilly, and no one was tracking any of it precisely… almost no one was thinking of it like they were getting momentary glimpses of a sort of state machine or advertising funnel or something, where each person might get a slightly different ride along a slightly different path AND ALSO might be at a different step along whatever ride they will end up having taken.
Another very common failing is that people think that a group mean implies a group median. If there are bimodal responses to something, and then a group summary is given… is the group’s denominator over “all treated” or “all who were treated and had a followup confirming an adequate response”?
BOTH of these errors have a common cause of “assuming homogeneous efficacy and assuming competent followup at the clinical level” and in my experience neither of these assumptions are strongly justified. They constantly fail, and people are constantly acting surprised about it.
Failures of followup are ALSO why we couldn’t get people properly quarantined at the beginning of this disaster.
Often patients DO NOT WANT to have “the system” FOLLOW them.
The study you linked to seems to have somewhat solved “the standard problems with the ‘lost to followup’ state” that is the bane of so many time course studies. The design certainly seems to take the followup process very much into account (and I couldn’t find drop out rates from skimming or ^f and so maybe literally no one dropped out):
Something I’d like to call attention to here… in the paper you link the Extended Track had Bleed1 data from the extended cohort group, where they got (in some sense) to see how many people seroconverted from just one dose by week 5 or so...
Recall that up in the abstract the paper summarizes the key result thusly:
Suppose, hypothetically, that instead of 91% of people having “a seroconverting response” on the first shot it was only 28% of them?
(This would be almost understandable. The youngest person in that study was 80 years old! The whole study is on a group whose immune systems should be assumed to be decrepit and fragile from the raw fact of great age.)
Then if the second exposure brought this up to 100% seroconversion “somehow”, and the seroconverted “antibody levels” were gaussian (log normal?) among the seroconverted and 0 among the rest...
...Then that bimodal response could directly and cleanly justify claiming “antibody response was 3.5-fold higher” in some very fuzzy and general way (because 28% x 3.5 = 98%)
MY NORMAL EXPECTATION is for people to communicate in a fuzzy and general way :-(
The graph you included as a supporting claim was, I think, just the B panel from the totality of Figure 2 which is nice in many ways. Color coded! The horizontal axes are mostly aligned! Nice!
Note that in Panel A the two timepoints give basically the same levels of antibody response, with maybe some hint of a slow decline, but also overlap with Panel B’s separated ranges. Some in Panel A went up?? Weird. Probably stuff goes down (and sometimes up?) over time, in general?
The data in Panel A therefore seems consistent to me that “eventually” there is some roughly normal and acceptable level of “vaccinated at all, in an essentially bimodal way” that two doses reaches faster than typical?
This is what the two dose shot is designed to do in my mind: get ALMOST ALL of the patients (because of herd immunity benefits) to the state of “CLEANLY SEROCONVERTED” with the LEAST amount of measurement and need for followup (because followup is really hard).
Bleed2 of the standard group is “10 weeks post standard dose2”. There is no Bleed3 for either group out all the way at week 21. That third data collection event would be “10 weeks after dose2 for the extended group” and thus sorta comparable to the standard group’s Bleed1?
My hunch is that extended Bleed3 would show a decline from the extended Bleed2 measurement…
...maybe this prediction is a crux?
I could also imagine those slow risers in the standard group would STILL be going up by week 21?
Basically, I suspect that antibody levels eventually go down EVENTUALLY (over months and years), but also have some “sensitivity to dynamics over a timecourse” (which is probably not showing up here, not because it didn’t happen, but because it wasn’t measured).
I don’t know. My error bars are wide.
...
Also it would have been great to measure antibody levels for everyone on week 3, as Bleed0? More of the dynamics would be visible I think, and it would help characterize (and separate?) various members of the standard group in terms of seroconversion status before the second dose?
...
Basically, immunological science is more of an art than a science. It has a gazillion moving parts created under extreme adversity. Also, humans underestimate the difficulties of just doing the thing over and over.
People don’t get that “seroconversion” IS A THING. Sometimes it just doesn’t happen. That’s often the most important practical fact. Many bad vaccine designs end up with a “vaccine” whose seroconversion rate is non-zero but so low as to be impractical. This meta-analysis for measles (trying to find a relationship between age and seroconversion) shows numerous things that were tried in the clinic that had less than 50% rates for some kids.
The Pfizer/Moderna/mRNA design is weirdly effective from my perspective here. To take a second dose of a weirdly effective thing… uh… I guess? Sure. If empirically that works for this disease maybe it could or would help somehow in a way that eventually could be made sense of.
So if I have to pick ONE THING to assume about vaccine status at the INDIVIDUAL level, it will be “seroconverted or not” and then after knowing the answer to that, I assume “the rest will be very very complicated”.
I freely admit this is a simplified model. I just also think that any specific additional mechanistic second step to the modeling effort is likely to explode the combinatorial space of patient states to worry about, and yet also be a TINY epicycle, and the first of MANY epicycles.
The thing I’m asking for is: what’s the best second epicycle to add? What is the mechanism? If someone is already seroconverted, what would you measure to detect “that their mechanistic biological state is not ALREADY in the configuration that you’d be hoping to cause to improve via the administration of a third dose”?
And in the meantime: DELTA MIGHT BE MUTATING AROUND THE CURRENT VACCINE DESIGN and so a new design aimed at the new epitopes just obviously seems like it would address the main uncertainty in a central way.
RadVac for Delta is something I might pay for, and maybe something I might want to try to make on my own? Searching a bit: it looks like mixed third dose trials are already starting, so… <3!
A third dose that’s “the same as the first two” doesn’t interest me. The second one doing what it does seems to be empirically real, but I don’t know why the hell it empirically gives the results it does.
On your “WHY”, you seem to be presenting reasons why other people not believing your model shouldn’t count as strong evidence against it. Which is all fair. But I’m still curious for positive evidence to believe your model in the first place. Maybe this would be obvious if I knew more biology, but as it is, I don’t know why I should place higher credence in your model than any other model (e.g. the one at the bottom of this comment, if that counts).
As far as I can tell, “antibody response was 3.5-fold higher” just means that, on average, people in the extended dosing schedule had 3.5x more antibodies. I can’t tell whether you interpret it in some other way, or if you think this is a misleading way to describe things, or if you’re making some other point...?
Yup!
Ok now I’m confused.
Do you think that all people on these graphs have reached a “normal and acceptable level of ‘vaccinated at all, in an essentially bimodal way’ ”?
If so, do you not think that there’s any important immunity difference between a single-vaccinated person around 1-10 on the graph, or a doubly-vaccinated person around 1000-10000?
Or if you think that only some of the people on this graph are immune, where do you think the line between immune and not-immune should be drawn on these graphs? (The distribution seems to be fairly continuous everywhere, to me, so it seems arbitrary to draw the line anywhere.)
Or if you think the important immunity difference isn’t captured by antibody-levels, what is it about?
And re “that two doses reaches faster than typical”; are you implying that the single-dosed people’s antibody response would’ve kept increasing beyond the 5-6 week mark and eventually gotten as high as the doubly-vaccinated people? That seems unlikely to me. (Other than maybe the few people where their antibodies did increase, but I’m happy to ignore them until I understand the most normal response curve better.)
Agreed.
Here’s one suggestion:
1. The more antibodies you have, the less probability of getting sick, the less probability of getting severe disease, etc.
2. More vaccines increases the number of antibodies you have.
3. Therefore you want to have more vaccines.
I would’ve thought (1) to be fairly uncontroversial? And the linked study seems to provide good evidence for (2) when going from 1 to 2 doses, increasing antibodies by roughly a factor of 100. And of course adding more vaccines will eventually stop adding more antibodies. But right now I don’t have any reason to believe in a big difference between going from 1->2 vaccines vs going from 2->3 vaccines (other than 2 vaccines being the general standard). So I wouldn’t be surprised if taking a 3rd vaccine could increase your antibodies by another order of magnitude.
Maybe you think this doesn’t provide enough of a “mechanism”? Biology being complicated, I’m very happy to take empirical data for what it is, and make extrapolations even if I don’t know what the mechanism is. Personally, I also don’t feel like I have any more mechanism for “vaccine have a fixed probability of causing antibodies if you don’t already have them, otherwise they don’t do much” than “vaccine typically increases antibodies by a lot regardless of whether you have them or not”. So when the evidence clearly indicates the latter, I will definitely believe it.
And yeah, also, if someone has the option, I agree that it seems probably better to get a different vaccine than the same vaccine again!
I sort of tapped out because “very long posts with an explosion of quotes” is a smell for me, but I wanted to continue because other indicators suggest “teaching and/or learning in good faith” <3
Finally posting now because of a big update from elsewhere...
For me, evidence happens at the point of measurement. Then often measurements are summarized in language by people who don’t think clearly, or worry about standard misinterpretations of simple measurements… so careful reading is sometimes required just to acquire evidence able to distinguish between models.
So for me, the default is to need to think about mechanistic timecourse evidence through the screen of “how it was confusingly explained to me” by people who often aren’t worried about mechanistic timecourse dynamics.
I kinda don’t care if people don’t believe my model, I just want my models to get better over time… and I’m happy to explain them to people, and I like teaching… but if people don’t believe me, then it is their tragedy that they believe false things, not my tragedy. (Conversely, people teaching me things is awesome!)
But to make my models better I don’t just import other people’s posterior believes about how a mechanistic system works, but rather see if my own model can “round trip” through my best guess of the raw data that they observed in a specific situation. If people have bad reasoning, then their posteriors are even less safe to import than otherwise...
FWIW, just tonight I got around to reading this cousin comment by Connor and it swiftly tipped me over almost entirely. Three doses… might work? Sure.
I already thought there were empirical reasons to think it, so for me I think the key words in Connor’s post started somewhere around:
The filter I have I think, is that I want to hear about mechanisms when it comes to biological theories.
I’m not saying button mashing doesn’t work. That plus “copy the winner” is how most actual technical innovation occurs and scales in practice most of the time. Its fine <3
But… a HUGE filter that avoids adding broken bits to my general reasoning capacities is whether someone can offer keywords that connects their proposed mechanism to ALL THE OTHER MECHANISMS in physics and chemistry and evolution and all of it.
Gimme a word like “IgA” and I can find my way to new and helpful parts of the truth mine! I can round trip it through general science, and so on.
I have paragraphs and paragraphs of text from my first attempt at a response, trying to explain “I don’t know and neither do you (but politely and at length)”.
They are deleted from this response. Maybe “two people debugging epistemics in the face of ignorance” is useful somehow for something, but I’m not attached to it. I could PM it maybe if you care?
Practical upshot: empirically more doses has worked, and now I have heard some “new magic mechanism words” from Connor, who seems to me to clearly knows his shit backwards and forwards and also seems to be in tentative favor of a third dose :-)
Maybe interesting: my main argument AGAINST a third dose is part of why I thought it might be smart to give single doses as fast as possible several months ago. Now that like… “mechanisms are mechanically different (giving more than just lots of IgM)” I feel like I learned enough to even notice errors in past thinking?
But also… weirdly(?) this same body of empirical results says that the second reaction works BETTER after … <missing mechanism that somehow is time dependent> has had 12 weeks to <do something> instead of just 3 weeks?
F-ing immunology, man. Its crazy.