Their previous version understated the results, since it didn’t include the uninfected. The new version does, and says (cutting out some text before and after, and emphasis added by me):
Methods: We invited individuals who were PCR tested for SARS-CoV-2 infection at participating hospitals between March 2020-November 2021 to fill an online questionnaire that included baseline demographics, details of their acute episode and information about symptoms they were currently experiencing. Using binomial regression, we compared vaccinated individuals with those unvaccinated and those uninfected in terms of self-reported symptoms post-acute infection.
Results: We included 951 infected and 2437 uninfected individuals. Of the infected, 637(67%) were vaccinated. The most commonly reported symptoms were; fatigue (22%), headache (20%), weakness (13%), and persistent muscle pain (10%). After adjusting for follow-up time and baseline symptoms, those who received two doses less likely than unvaccinated individuals to report any of these symptoms by 64%, 54%, 57%, and 68% respectively, (Risk ratios 0.36, 0.46, 0.43, 0.32, p<0.04 in the listed sequence). Those who received two doses were no more likely to report any of these symptoms than individuals reporting no previous SARS-CoV-2 infection.
Conclusions: Vaccination with at least two doses of COVID-19 vaccine was associated with a substantial decrease in reporting the most common post-acute COVID-19 symptoms, bringing it back to baseline. Our results suggest that, in addition to reducing the risk of acute illness, COVID-19 vaccination may have a protective effect against long COVID.
I think all groups here consisted of infected individuals, except for “Uninfected”
One thing that’s pretty suspicious here and could cast doubt on these results is that the uninfected were more likely than the infected doubly vaccinated to have some of these long COVID symptoms, including the most common one, fatigue (and that seems to be the only one for which there’s a statistically significant difference, driven by those aged 36+).
On what they did:
A first series of binomial regression models were fitted to the data for the ten most commonly reported post-COVID-19 symptoms according to vaccination status. We adjusted for the difference in follow-up time and proportion of asymptomatic patients at the time of diagnosis between the groups. We then compared vaccinated and infected individuals to never infected individuals in terms of reported symptoms, also using binomial regression models. We used two distinct steps since it was not possible to adjust for follow-up time and symptoms at baseline among uninfected patients.
To take the anticipated age differences into account, the analysis was age-stratified and differences in the length of time from the beginning of symptoms to responding to the survey were adjusted for in the model. Vaccination status was recorded as either one dose or two doses. At the time of data collection, very few individuals had received a third dose and those who did were recorded as two doses.
It’s not clear to me whether or not they adjusted for the demographic characteristics they collected (summarized in table 1).
Some possible explanations and potential weaknesses of the study:
1. The sample sizes are not very large, so it could be noisy. They only had 59, 135 and 100 doubly vaccinated + infected individuals in the 19-35, 36-60 and >60 age groups, respectively. The number of uninfected was much larger, though, 2437 together. As far as I can tell, they didn’t make any adjustments for multiple comparison tests, and they did >160 tests, for the different symptoms and age groups.
2. I don’t know if they made any adjustments (other than age stratification) for the comparisons to the Uninfected group, and this could leave bias. If they did make any such adjustments, they might not have made enough. They didn’t do any matching.
There could be group differences not accounted for that would bias it this way, e.g. people who got vaccinated and infected could have been more health-conscious on average than those who were uninfected (with any vaccination status). We might think the uninfected are demographically between the unvaccinated+infected and vaccinated+infected. Compared to the unvaccinated+infected, the doubly vaccinated+infected were more likely to have tertiary education (62.9% vs 52.1%), more likely to have hypertension (13.9% vs 5.1%) and more likely to have diabetes (8.2% vs 3.8%). Hypertension and diabetes should point to the opposite bias, though, if unadjusted for. (I think it’s unlikely they adjusted for anything they should not have adjusted for in a way that increased bias, although such adjustments can increase variance.)
3. Some of those considered uninfected could actually have been infected, and ended up with long COVID. They used “uninfected in terms of self-reported symptoms post-acute infection”, but it’s possible some asymptomatic infected individuals get long COVID, and some of these asymptomatic infected individuals were probably not vaccinated. Their positive cases were based on “self-reported positive PCR results”.
4.The vaccines could protect against long COVID-like symptoms from other causes. Maybe through the placebo effect?
Overall, given how large the gap is between uninfected and doubly vaccinated+infected, and the reasonable assumption that vaccination + infection should actually be worse than never getting infected, that gap could easily hide a risk of long COVID of around 1% (at least for fatigue).
I think we should look further into this study, which seems somewhat reassuring, but I have reservations about it:
Kuodi et al., January 2022, “Association between vaccination status and reported incidence of post-acute COVID-19 symptoms in Israel: a cross-sectional study of patients tested between March 2020 and November 2021” (pdf, not yet peer-reviewed)
Their previous version understated the results, since it didn’t include the uninfected. The new version does, and says (cutting out some text before and after, and emphasis added by me):
One thing that’s pretty suspicious here and could cast doubt on these results is that the uninfected were more likely than the infected doubly vaccinated to have some of these long COVID symptoms, including the most common one, fatigue (and that seems to be the only one for which there’s a statistically significant difference, driven by those aged 36+).
On what they did:
It’s not clear to me whether or not they adjusted for the demographic characteristics they collected (summarized in table 1).
Some possible explanations and potential weaknesses of the study:
1. The sample sizes are not very large, so it could be noisy. They only had 59, 135 and 100 doubly vaccinated + infected individuals in the 19-35, 36-60 and >60 age groups, respectively. The number of uninfected was much larger, though, 2437 together. As far as I can tell, they didn’t make any adjustments for multiple comparison tests, and they did >160 tests, for the different symptoms and age groups.
2. I don’t know if they made any adjustments (other than age stratification) for the comparisons to the Uninfected group, and this could leave bias. If they did make any such adjustments, they might not have made enough. They didn’t do any matching.
There could be group differences not accounted for that would bias it this way, e.g. people who got vaccinated and infected could have been more health-conscious on average than those who were uninfected (with any vaccination status). We might think the uninfected are demographically between the unvaccinated+infected and vaccinated+infected. Compared to the unvaccinated+infected, the doubly vaccinated+infected were more likely to have tertiary education (62.9% vs 52.1%), more likely to have hypertension (13.9% vs 5.1%) and more likely to have diabetes (8.2% vs 3.8%). Hypertension and diabetes should point to the opposite bias, though, if unadjusted for. (I think it’s unlikely they adjusted for anything they should not have adjusted for in a way that increased bias, although such adjustments can increase variance.)
3. Some of those considered uninfected could actually have been infected, and ended up with long COVID. They used “uninfected in terms of self-reported symptoms post-acute infection”, but it’s possible some asymptomatic infected individuals get long COVID, and some of these asymptomatic infected individuals were probably not vaccinated. Their positive cases were based on “self-reported positive PCR results”.
4.The vaccines could protect against long COVID-like symptoms from other causes. Maybe through the placebo effect?
Overall, given how large the gap is between uninfected and doubly vaccinated+infected, and the reasonable assumption that vaccination + infection should actually be worse than never getting infected, that gap could easily hide a risk of long COVID of around 1% (at least for fatigue).