It’s tricky to run studies with beliefs as a variable, because beliefs have causes, so you’re setting yourself up to have confounds. I haven’t looked that closely at this study, but here are some possibilities:
Severity: people who had covid but believed that they didn’t had mild symptoms. So ‘severe cases have more long-term symptoms than mild/asymptomatic cases’ would look like ‘covid+belief leads to more reported long-term symptoms than covid without belief’.
Other illnesses: people who didn’t have covid but thought the did had some other illness like flu or pneumonia. If there’s long flu, then the long-term symptoms could be from that.
Long-term symptoms: a person who thinks that they probably just have a cold and not covid, but then is still fatigued a month later, might conclude that actually it probably was covid. So medium-to-long-term symptoms can cause belief, rather than belief causing long-term symptoms.
Testing inaccuracy: if the test that they’re using to establish the ground truth of whether a person had covid isn’t that accurate, then people who they’re counting as ‘covid but no belief’ might actually be false positives, and people who they’re counting as ‘no covid but yes belief’ might be false negatives.
Hypochondria: people who are to imagine that their health is worse than it actually is might mistakenly believe that they had covid (when they didn’t) and also imagine that they have long-term symptoms like fatigue or difficulty breathing. If people who did get covid have similar reported long-term symptoms, that means that the actual long-term symptoms of people who had covid are as bad as the imagined level of symptoms among people who imagined they had covid.
Denial: the reverse of hypochondria—people who say they’re fine even when they have some health symptoms might say that they didn’t have covid even though they did, and then downplay their long-term symptoms.
Trolling: if data slipped into the study from any people who find it funny to give extreme answers, they would look like hypochondriacs, claiming to have covid & long-term symptoms even if they didn’t have covid.
The first few of these possibilities are cases where facts about the world influence beliefs, and those facts also influence long-term symptoms. The last few of these possibilities are where the person’s traits influence their beliefs (or stated beliefs), and those traits also influence their reports of what long-term symptoms they’ve had.
If you wanted to independently assess the effects of getting covid and believing that you had covid, ideally (for scientific rigor unconstrained by ethics or practicality) you’d randomly assign some people to get covid or not and also randomly assign some people to believe they had covid or not (e.g. by lying to them). If you couldn’t have perfect random assignment + blinding, then you’d want to measure a whole bunch of other variables to account for them statistically. In reality, without anything like random assignment, who gets covid is maybe close enough to random for an observational study to work, especially if you control for some high-level variables like age. Beliefs about whether you had covid are heavily entangled with relevant stuff, in a way that makes it really hard to study them as an independent variable.
Is there good reason to think that this study overcomes these problems?
For comparison: imagine some medical researchers are interested in whether a particular medicine helps with a particular medical condition, so they set up a placebo controlled trial. A bunch of people with the medical condition all get their symptoms tested, then they flip a coin and half get pills with the medicine while the other half get sugar pills, and they don’t know whether they have the real pills. Then, some time later, they all get their symptoms tested again.
Now, imagine that I’m interested in “placebo effects”—I want to see if the ritual of taking sugar pills which you think might be medicine improves people’s health, or causes side effects, and I want to piggyback on this medical trial. I could just look at the pre vs post results for the set of people who got the sugar pills, but unfortunately this medical condition varies over time so I can’t disentangle effects of the pill-taking ritual from changes over time. I wish the study had a third “no-pill” group who (knowingly) didn’t get any treatment, in addition to the medical pill group and the inert pill group. Then I could just compare the results of the sugar pill group to the no pill group. But it doesn’t.
So I have the clever idea of getting the researchers to add a question to the tests at the end of the study, where they ask the patients whether they think they got the medicine pill or the sugar pill. That gives me a nice 2x2 design, where patients differ both in whether they got the medicine pill or the sugar pill, and separately in whether they believe they got the medicine pill or the sugar pill. So I can look separately at each of the 4 groups to see how much their condition improved, and what side effects they got. Changes that are associated with beliefs, I can claim, are based on the psychological effects of this pill taking ritual rather than the physiological effects of the substance they ingested.
This is a terrible study design. Who’s going to believe they got the real medicine? Well, people whose condition improved will tend to think they must’ve gotten the real medicine. And people who noticed physiological states like nausea or dry mouth will tend to think they’ve gotten the real medicine. This study design will say that improved condition & nausea are caused by people’s beliefs about whether they got the medicine, when in reality it’s the reverse: the beliefs are caused by these physical changes.
If I’m especially meddlesome, I might even tell the original researchers that they should use this 2x2 design to evaluate the original study. Instead of just comparing the outcomes for the medicine pill group and the sugar pill group, they should compare the outcomes while controlling for people’s beliefs about whether they got the medicine. That would mess up their study. It would be asking how effective the medicine is, after removing any effects that allowed patients to realize that they’d gotten the medicine (as if belief-entangled effects couldn’t be physiological effects of the substance).
It’s tricky to run studies with beliefs as a variable, because beliefs have causes, so you’re setting yourself up to have confounds. I haven’t looked that closely at this study, but here are some possibilities:
Severity: people who had covid but believed that they didn’t had mild symptoms. So ‘severe cases have more long-term symptoms than mild/asymptomatic cases’ would look like ‘covid+belief leads to more reported long-term symptoms than covid without belief’.
Other illnesses: people who didn’t have covid but thought the did had some other illness like flu or pneumonia. If there’s long flu, then the long-term symptoms could be from that.
Long-term symptoms: a person who thinks that they probably just have a cold and not covid, but then is still fatigued a month later, might conclude that actually it probably was covid. So medium-to-long-term symptoms can cause belief, rather than belief causing long-term symptoms.
Testing inaccuracy: if the test that they’re using to establish the ground truth of whether a person had covid isn’t that accurate, then people who they’re counting as ‘covid but no belief’ might actually be false positives, and people who they’re counting as ‘no covid but yes belief’ might be false negatives.
Hypochondria: people who are to imagine that their health is worse than it actually is might mistakenly believe that they had covid (when they didn’t) and also imagine that they have long-term symptoms like fatigue or difficulty breathing. If people who did get covid have similar reported long-term symptoms, that means that the actual long-term symptoms of people who had covid are as bad as the imagined level of symptoms among people who imagined they had covid.
Denial: the reverse of hypochondria—people who say they’re fine even when they have some health symptoms might say that they didn’t have covid even though they did, and then downplay their long-term symptoms.
Trolling: if data slipped into the study from any people who find it funny to give extreme answers, they would look like hypochondriacs, claiming to have covid & long-term symptoms even if they didn’t have covid.
The first few of these possibilities are cases where facts about the world influence beliefs, and those facts also influence long-term symptoms. The last few of these possibilities are where the person’s traits influence their beliefs (or stated beliefs), and those traits also influence their reports of what long-term symptoms they’ve had.
If you wanted to independently assess the effects of getting covid and believing that you had covid, ideally (for scientific rigor unconstrained by ethics or practicality) you’d randomly assign some people to get covid or not and also randomly assign some people to believe they had covid or not (e.g. by lying to them). If you couldn’t have perfect random assignment + blinding, then you’d want to measure a whole bunch of other variables to account for them statistically. In reality, without anything like random assignment, who gets covid is maybe close enough to random for an observational study to work, especially if you control for some high-level variables like age. Beliefs about whether you had covid are heavily entangled with relevant stuff, in a way that makes it really hard to study them as an independent variable.
Is there good reason to think that this study overcomes these problems?
For comparison: imagine some medical researchers are interested in whether a particular medicine helps with a particular medical condition, so they set up a placebo controlled trial. A bunch of people with the medical condition all get their symptoms tested, then they flip a coin and half get pills with the medicine while the other half get sugar pills, and they don’t know whether they have the real pills. Then, some time later, they all get their symptoms tested again.
Now, imagine that I’m interested in “placebo effects”—I want to see if the ritual of taking sugar pills which you think might be medicine improves people’s health, or causes side effects, and I want to piggyback on this medical trial. I could just look at the pre vs post results for the set of people who got the sugar pills, but unfortunately this medical condition varies over time so I can’t disentangle effects of the pill-taking ritual from changes over time. I wish the study had a third “no-pill” group who (knowingly) didn’t get any treatment, in addition to the medical pill group and the inert pill group. Then I could just compare the results of the sugar pill group to the no pill group. But it doesn’t.
So I have the clever idea of getting the researchers to add a question to the tests at the end of the study, where they ask the patients whether they think they got the medicine pill or the sugar pill. That gives me a nice 2x2 design, where patients differ both in whether they got the medicine pill or the sugar pill, and separately in whether they believe they got the medicine pill or the sugar pill. So I can look separately at each of the 4 groups to see how much their condition improved, and what side effects they got. Changes that are associated with beliefs, I can claim, are based on the psychological effects of this pill taking ritual rather than the physiological effects of the substance they ingested.
This is a terrible study design. Who’s going to believe they got the real medicine? Well, people whose condition improved will tend to think they must’ve gotten the real medicine. And people who noticed physiological states like nausea or dry mouth will tend to think they’ve gotten the real medicine. This study design will say that improved condition & nausea are caused by people’s beliefs about whether they got the medicine, when in reality it’s the reverse: the beliefs are caused by these physical changes.
If I’m especially meddlesome, I might even tell the original researchers that they should use this 2x2 design to evaluate the original study. Instead of just comparing the outcomes for the medicine pill group and the sugar pill group, they should compare the outcomes while controlling for people’s beliefs about whether they got the medicine. That would mess up their study. It would be asking how effective the medicine is, after removing any effects that allowed patients to realize that they’d gotten the medicine (as if belief-entangled effects couldn’t be physiological effects of the substance).