John Ioannidis is making an interesting (and reassuring, if true) claim here. Has anyone looked at the demographics and done the comparison themselves?
This can’t be right. I’ve looked into Diamond cruise studies and some stay at 1% even after adjusting for age (they factor in that more people might have died in the meantime – even though that didn’t happen so far, admittedly – and the unadjusted number is 1% at least already; makes you wonder whether elderly cruise goers are healthier than their stay-at-home cohorts). I’ve found this study which, after doing some adjustment steps I don’t understand but find dubious (maybe they double adjusted something by accident?), ends up estimating 0.5% for China’s total outbreak. You might think this makes the 0.125% figure mentioned by Joannidis somewhat plausible, because China’s outbreak had a majority of cases in Hubei where patients didn’t all have access to hospital care. This is likely to drive up the fatality rate. However, the study didn’t account for that. They just implicitly assumed that people who got sick on the cruise ship had the same prospects as people who got sick in China. And they may or may not have halved their estimates in some dubious way too. So, 0.5% seems like an absolute lower bound here, and more likely it’s higher. 0.125 is extremely implausible if you ask me.
UPDATE: I did some calculations for age adjustments on the Diamond Princess here and I now consider it to be only weak evidence against a 0.125% estimate. My own age adjustment returned a point estimate of about 0.2%, which I argue should be further adjusted upward for reasons related to selection effects of the type of people who go on cruises.
In addition, South Korea now has a 1% CFR. They’ve done >230,000 tests, sometimes more than 10,000 tests per day. They only have about 8,500 confirmed diagnoses, so with the number of tests that they’re doing, one would assume that they caught most of the illnesses. I think they even must have caught most of the asymptomatic cases, because a large portion of their diagnoses was from this Christian sect (which happens to have a young demographic too), and I think they tested almost everyone on their membership records.
Finally, the Imperial College experts who advise the UK government recently revised their estimates and gave out a 0.9% infection fatality rate estimate (see page 5 mostly). This presumably applies to favorable conditions rather than hospital overstrain. Edit: In the 80k podcast Howie Lempel said it’s a prediction about what’s likely to hit the UK. He might be right and if that’s the case, it would factor in some degree of hospital overstrain.
All of this strongly suggests that Joannidis is spreading dangerous misinformation. But maybe there’s something I and others are not seeing.
Yeah, the 1/8th multiplier sounded hard to believe. A 1⁄2 multiplier based on demographic correction sounds a lot more plausible, and it’s nice to have confirmation that someone else actually did the math. Thanks for finding/sharing it!
I should flag that I didn’t do the math for age correction. I only got this from another Diamond cruise study where the age correction provided a smaller update (and I didn’t really like other things about that study).
So, I think it could be valuable to investigate this claim more:
Projecting the Diamond Princess mortality rate onto the age structure of the U.S. population, the death rate among people infected with Covid-19 would be 0.125%.
(But even if this point was right, there’s still South Korea to explain.)
Reading the Ioannidis article, it seems to say that he did his own calculations, and he doesn’t show them. Okay.
I’m curious about this, so I’m going to try a ballparking estimate myself.
Tl;dr I intially arrived at a result that suggested 0.125% was way off, but then found better info on the cruise ship’s age distribution and had to revise my judgment. I now find it debatable whether 0.125% is defensible or not, but it’s not “way off.” My own estimate would be more in the ballpark of 0.3%, but I don’t anymore consider the cruise ship to be evidence for IFR estimates at 0.5% or higher.
Update March 24th: In the couple of days, 3 new patients who had tested positive on the Diamond Princess have died. In addition, the Wikipedia article has been edited to list another death that previously hadn’t been included. So total deaths per confirmed cases on the Diamond Princess are now 11 / 700 instead of 7 / 700. All my calculations below are based on the older, outdated numbers. To get the most updated estimates, just multiply the results below by 11⁄7.
---
Note that I have never done age adjustments for anything, so I have no idea what the proper methdology would be. I’m just curious to see if 0.125% is potentially reasonable rather than (as my current intuition suggests) very dubious.
A total of 634 people tested positive among 3,063 tests as at 20 February 2020. Of 634 cases, a total of 313 cases were female and six were aged 0–19 years, 152 were aged 20–59 years and 476 were 60 years and older.
At the end of the outbreak, roughly 700 people had tested positive. I’m going to assume that the 66 patients not yet in the above statistics fall into age categories in the same proportion. So a bit more than two thirds of the 66 patients get added to the 476 figure for people aged 60 and older.
With this adjustment, we have 700 diagnosed cases, of which an estimated 525 patients were aged 60 and older. Of those 700 diagnosed cases, 7 people died. 525 out of 700 corresponds to 75%. (I’m going to mostly ignore the death risk for people below age 60 for the analysis below, because it will be negligible given that people older than that anyway make up the majority share.)
Eyeballing this, let’s go with 22% of the population at age 60 or older.
75 divided by 22 is roughly 3.4, so this naively suggests that the cruise ship’s demographic was roughly 3.4 times more susceptible to dying from SARS-CoV-2. If I divide the observed IFR of 1% by 3.4, I get 0.3%. Why does Ioannidis get 0.125% instead of 0.3?
Moreover, it seems to me that 0.3% must be an underestimate because I assume that even though the cruise ship population is substantially older on average than the US population, I would think that this effect will disappear (or even reverse) at the extremes, once we look at the percentage of exceptionally old people (e.g., aged 80 and above, age 85 and above, etc.). Because Covid-19 is particularly fatal for the very oldest people, I expect the 0.3% figure to contain a substantial degree of overcorrection. Especially also because elderly people with the most severe pre-existing health conditions are likely heavily underrepresented on cruise ships. This effect could be really quite significant: It’s not even totally obvious that a downward adjustment of the 1% IFR observed on the Diamond Princess is warranted. It’s probably warranted, but depending on how strongly cruise ship passengers are pre-selected against having unusually bad health, and depending on how strongly pre-existing health conditions affect someone’s survival prospect for Covid-19, it’s conceivable that the 1% figure doesn’t need to be downward adjusted at all.
To conclude, I don’t understand how age adjustments for SARS-Cov-2 infections on the Diamond Princess can drive down the estimated IFR substantially below 0.5%. 0.5% seems closer to a lower bound to me than anything else. (Of course, those are point estimates. I don’t have strong views on whether 0.125% is outside some appropriate confidence interval, but my impression was that 0.125% was Ioannidis’s point estimate, and interpreted as such, it seems clearly much too low!)
UPDATE: Oh I see. I found an age table that I had overlooked initially. It turns out cruises are really popular for people at age 70-79 (there are about 20% more people of that age than 60-69, whereas it’s the other way around for US demographics). This distribution makes Ioannidis’s figures look more plausible, though the difference doesn’t seem large enough to fully bridge the gap between 0.3% and 0.125%, especially because the 80-89 bracket seems to be represented proportionally again. Still, I don’t anymore think that 0.125% is horribly off.
You found an age distribution for the infected population on the Diamond Princess, but you’re using it as if it’s the age distribution for everyone on the ship. Older people are more likely to get infected, so the infected population in the US will lean older as well—closer to the distribution on the ship. To do a good age adjustment we need to know the ages of the people on the ship who were not infected.
Older people are more likely to get infected, so the infected population in the US will lean older as well—closer to the distribution on the ship.
Interesting! Do you think this is established? I haven’t looked into this, but my guess would have been that the risk is similar because young people are less scared of the virus. But yeah, good point about further adjustments being needed to get the best estimate.
Hmm, maybe you’re right. The South Korean distribution of cases by age here suggests that it’s actually most common by far among people in their twenties, and the larger number of confirmed cases among older people is a statistical artifact resulting from test criteria. The data do look a bit suspicious though.
Right, I got that it was them doing the math correction not you. Still, they did the math and give an age breakdown of the passengers and a crude sanity check gives a number within about 30% of what they report.
I don’t think the view in that piece here is consistent with what happened in Lombardia in Italy, but I haven’t seen a detailed numerical argument against it.
I also thought that in Lombardia, the estimates given by Ioannidis are rapidly trending toward coming in contradiction with SIR models. :( Lombardia has a population of 11 million people and 2,500 reported deaths. Some doctors are raising alarm that many deaths are going undetected because people are dying at a rate that’s 4 times higher than the same month last year. In addition, the death counts always lags behind because some people are sick for a long time before they die (though maybe this start to be the case less strongly in conditions of extreme hospital overstrain). All of this suggests that an estimate of 10,000 deaths for Lombardia alone might soon prove to be accurate. But according to the IFR provided by Ioannidis, this would correspond to an expected 8 million people infected (72% of the population). I don’t understand SIR models well enough to calculate what the R0 would have to be for 72% of a population to get infected. I suspect that Covid-19′s R0 is high enough to be consistent with this, but it wouldn’t leave a lot of room for estimation errors.
That said, I think the above calculation is naive, so the argument doesn’t work (at least not in this crude form). If hospitals become as overwhelmed as they are in Italy, I’m sure that even someone with Ioannidis’ view would expect the IFR for Lombardy to become a lot higher than 0.125% because a lot of people aren’t getting life-saving hospital attention.
So, this means that Lombardy isn’t necessarily a knockdown argument against Ioannidis’s estimate in the same way South Korea is. However, I think Ioannidis’s estimate would have counterintuitive implications for the percentage of people infected in Lombardy. It would have to be in the double digits already at the very least. The most trustworthy estimate I saw about Wuhan suggested that only 5% of its population had the virus. However, there’s some disagreement about this, and the people who tend to argue for an unusually low IFR also tend to argue that there’s a giant iceberg of undetected asymptomatic cases.
UPDATE: I just realized something: I read somewhere recently that Italy is doing 30,000 tests a day by now, and that about 25% of them are positive. This seems to be in contradiction with Ioannidis’s estimate because his view should imply that, if there’s some kind of selection at all for who they are testing (as opposed to just testing members of the population at random), then we should expect to see more positive test results than 25%. (Why? Because if we assume that hospital overstrain increases his death rate estimate by a factor of 7x (which is a really large adjustment!), the death count estimates for Lombardy combined with Ioannidis’s estimates would still suggest that above 10% of the population would have the virus. Such high numbers would only be consistent with reality if most people had relatively mild symptoms or no symptoms at all, so assuming that there’s substantial pre-selection on who is getting tested (as opposed to random testing, which would be odd), a rate of 25% positive tests would be implausibly low for the scenario where >10% of the region were infected. So, to conclude, I think one can plausibly construct a case against Ioannidis’s estimates based solely on common sense and numbers from Lombardia. I probably haven’t quite succeeded at making this case in a watertight way, but I think you might be right with your intuition. This is just one more reason why the 0.125% estimate is completely absurd.
John Ioannidis is making an interesting (and reassuring, if true) claim here. Has anyone looked at the demographics and done the comparison themselves?
This can’t be right. I’ve looked into Diamond cruise studies and some stay at 1% even after adjusting for age (they factor in that more people might have died in the meantime – even though that didn’t happen so far, admittedly – and the unadjusted number is 1% at least already; makes you wonder whether elderly cruise goers are healthier than their stay-at-home cohorts). I’ve found this study which, after doing some adjustment steps I don’t understand but find dubious (maybe they double adjusted something by accident?), ends up estimating 0.5% for China’s total outbreak. You might think this makes the 0.125% figure mentioned by Joannidis somewhat plausible, because China’s outbreak had a majority of cases in Hubei where patients didn’t all have access to hospital care. This is likely to drive up the fatality rate. However, the study didn’t account for that. They just implicitly assumed that people who got sick on the cruise ship had the same prospects as people who got sick in China. And they may or may not have halved their estimates in some dubious way too. So, 0.5% seems like an absolute lower bound here, and more likely it’s higher. 0.125 is extremely implausible if you ask me.
UPDATE: I did some calculations for age adjustments on the Diamond Princess here and I now consider it to be only weak evidence against a 0.125% estimate. My own age adjustment returned a point estimate of about 0.2%, which I argue should be further adjusted upward for reasons related to selection effects of the type of people who go on cruises.
In addition, South Korea now has a 1% CFR. They’ve done >230,000 tests, sometimes more than 10,000 tests per day. They only have about 8,500 confirmed diagnoses, so with the number of tests that they’re doing, one would assume that they caught most of the illnesses. I think they even must have caught most of the asymptomatic cases, because a large portion of their diagnoses was from this Christian sect (which happens to have a young demographic too), and I think they tested almost everyone on their membership records.
Finally, the Imperial College experts who advise the UK government recently revised their estimates and gave out a 0.9% infection fatality rate estimate (see page 5 mostly). This presumably applies to favorable conditions rather than hospital overstrain. Edit: In the 80k podcast Howie Lempel said it’s a prediction about what’s likely to hit the UK. He might be right and if that’s the case, it would factor in some degree of hospital overstrain.
All of this strongly suggests that Joannidis is spreading dangerous misinformation. But maybe there’s something I and others are not seeing.
Yeah, the 1/8th multiplier sounded hard to believe. A 1⁄2 multiplier based on demographic correction sounds a lot more plausible, and it’s nice to have confirmation that someone else actually did the math. Thanks for finding/sharing it!
I should flag that I didn’t do the math for age correction. I only got this from another Diamond cruise study where the age correction provided a smaller update (and I didn’t really like other things about that study).
So, I think it could be valuable to investigate this claim more:
(But even if this point was right, there’s still South Korea to explain.)
Reading the Ioannidis article, it seems to say that he did his own calculations, and he doesn’t show them. Okay.
I’m curious about this, so I’m going to try a ballparking estimate myself.
Tl;dr I intially arrived at a result that suggested 0.125% was way off, but then found better info on the cruise ship’s age distribution and had to revise my judgment. I now find it debatable whether 0.125% is defensible or not, but it’s not “way off.” My own estimate would be more in the ballpark of 0.3%, but I don’t anymore consider the cruise ship to be evidence for IFR estimates at 0.5% or higher.
Update March 24th: In the couple of days, 3 new patients who had tested positive on the Diamond Princess have died. In addition, the Wikipedia article has been edited to list another death that previously hadn’t been included. So total deaths per confirmed cases on the Diamond Princess are now 11 / 700 instead of 7 / 700. All my calculations below are based on the older, outdated numbers. To get the most updated estimates, just multiply the results below by 11⁄7.
---
Note that I have never done age adjustments for anything, so I have no idea what the proper methdology would be. I’m just curious to see if 0.125% is potentially reasonable rather than (as my current intuition suggests) very dubious.
From this paper, I found the following info:
At the end of the outbreak, roughly 700 people had tested positive. I’m going to assume that the 66 patients not yet in the above statistics fall into age categories in the same proportion. So a bit more than two thirds of the 66 patients get added to the 476 figure for people aged 60 and older.
With this adjustment, we have 700 diagnosed cases, of which an estimated 525 patients were aged 60 and older. Of those 700 diagnosed cases, 7 people died. 525 out of 700 corresponds to 75%. (I’m going to mostly ignore the death risk for people below age 60 for the analysis below, because it will be negligible given that people older than that anyway make up the majority share.)
This wikipedia article on US demographics says the following:
0–14 years: 18.62%
15–24 years: 13.12%
25–54 years: 39.29%
55–64 years: 12.94%
65 years and over: 16.03%
Eyeballing this, let’s go with 22% of the population at age 60 or older.
75 divided by 22 is roughly 3.4, so this naively suggests that the cruise ship’s demographic was roughly 3.4 times more susceptible to dying from SARS-CoV-2. If I divide the observed IFR of 1% by 3.4, I get 0.3%. Why does Ioannidis get 0.125% instead of 0.3?
Moreover, it seems to me that 0.3% must be an underestimate because I assume that even though the cruise ship population is substantially older on average than the US population, I would think that this effect will disappear (or even reverse) at the extremes, once we look at the percentage of exceptionally old people (e.g., aged 80 and above, age 85 and above, etc.). Because Covid-19 is particularly fatal for the very oldest people, I expect the 0.3% figure to contain a substantial degree of overcorrection. Especially also because elderly people with the most severe pre-existing health conditions are likely heavily underrepresented on cruise ships. This effect could be really quite significant: It’s not even totally obvious that a downward adjustment of the 1% IFR observed on the Diamond Princess is warranted. It’s probably warranted, but depending on how strongly cruise ship passengers are pre-selected against having unusually bad health, and depending on how strongly pre-existing health conditions affect someone’s survival prospect for Covid-19, it’s conceivable that the 1% figure doesn’t need to be downward adjusted at all.
To conclude, I don’t understand how age adjustments for SARS-Cov-2 infections on the Diamond Princess can drive down the estimated IFR substantially below 0.5%. 0.5% seems closer to a lower bound to me than anything else. (Of course, those are point estimates. I don’t have strong views on whether 0.125% is outside some appropriate confidence interval, but my impression was that 0.125% was Ioannidis’s point estimate, and interpreted as such, it seems clearly much too low!)
UPDATE: Oh I see. I found an age table that I had overlooked initially. It turns out cruises are really popular for people at age 70-79 (there are about 20% more people of that age than 60-69, whereas it’s the other way around for US demographics). This distribution makes Ioannidis’s figures look more plausible, though the difference doesn’t seem large enough to fully bridge the gap between 0.3% and 0.125%, especially because the 80-89 bracket seems to be represented proportionally again. Still, I don’t anymore think that 0.125% is horribly off.
You found an age distribution for the infected population on the Diamond Princess, but you’re using it as if it’s the age distribution for everyone on the ship. Older people are more likely to get infected, so the infected population in the US will lean older as well—closer to the distribution on the ship. To do a good age adjustment we need to know the ages of the people on the ship who were not infected.
Interesting! Do you think this is established? I haven’t looked into this, but my guess would have been that the risk is similar because young people are less scared of the virus. But yeah, good point about further adjustments being needed to get the best estimate.
Hmm, maybe you’re right. The South Korean distribution of cases by age here suggests that it’s actually most common by far among people in their twenties, and the larger number of confirmed cases among older people is a statistical artifact resulting from test criteria. The data do look a bit suspicious though.
Right, I got that it was them doing the math correction not you. Still, they did the math and give an age breakdown of the passengers and a crude sanity check gives a number within about 30% of what they report.
I don’t think the view in that piece here is consistent with what happened in Lombardia in Italy, but I haven’t seen a detailed numerical argument against it.
I also thought that in Lombardia, the estimates given by Ioannidis are rapidly trending toward coming in contradiction with SIR models. :( Lombardia has a population of 11 million people and 2,500 reported deaths. Some doctors are raising alarm that many deaths are going undetected because people are dying at a rate that’s 4 times higher than the same month last year. In addition, the death counts always lags behind because some people are sick for a long time before they die (though maybe this start to be the case less strongly in conditions of extreme hospital overstrain). All of this suggests that an estimate of 10,000 deaths for Lombardia alone might soon prove to be accurate. But according to the IFR provided by Ioannidis, this would correspond to an expected 8 million people infected (72% of the population). I don’t understand SIR models well enough to calculate what the R0 would have to be for 72% of a population to get infected. I suspect that Covid-19′s R0 is high enough to be consistent with this, but it wouldn’t leave a lot of room for estimation errors.
That said, I think the above calculation is naive, so the argument doesn’t work (at least not in this crude form). If hospitals become as overwhelmed as they are in Italy, I’m sure that even someone with Ioannidis’ view would expect the IFR for Lombardy to become a lot higher than 0.125% because a lot of people aren’t getting life-saving hospital attention.
So, this means that Lombardy isn’t necessarily a knockdown argument against Ioannidis’s estimate in the same way South Korea is. However, I think Ioannidis’s estimate would have counterintuitive implications for the percentage of people infected in Lombardy. It would have to be in the double digits already at the very least. The most trustworthy estimate I saw about Wuhan suggested that only 5% of its population had the virus. However, there’s some disagreement about this, and the people who tend to argue for an unusually low IFR also tend to argue that there’s a giant iceberg of undetected asymptomatic cases.
UPDATE: I just realized something: I read somewhere recently that Italy is doing 30,000 tests a day by now, and that about 25% of them are positive. This seems to be in contradiction with Ioannidis’s estimate because his view should imply that, if there’s some kind of selection at all for who they are testing (as opposed to just testing members of the population at random), then we should expect to see more positive test results than 25%. (Why? Because if we assume that hospital overstrain increases his death rate estimate by a factor of 7x (which is a really large adjustment!), the death count estimates for Lombardy combined with Ioannidis’s estimates would still suggest that above 10% of the population would have the virus. Such high numbers would only be consistent with reality if most people had relatively mild symptoms or no symptoms at all, so assuming that there’s substantial pre-selection on who is getting tested (as opposed to random testing, which would be odd), a rate of 25% positive tests would be implausibly low for the scenario where >10% of the region were infected. So, to conclude, I think one can plausibly construct a case against Ioannidis’s estimates based solely on common sense and numbers from Lombardia. I probably haven’t quite succeeded at making this case in a watertight way, but I think you might be right with your intuition. This is just one more reason why the 0.125% estimate is completely absurd.