Notalgebraist posted a reasonable critique of Flattening The Curve Is a Deadly Delusion, explaining how it’s incorrect to assume that flattenings won’t reduce the total number of cases, and that it doesn’t make sense to assume a normal distribution. I think that the the piece’s main point was correct, though: that the curve would need to get crazy flat for hospitals to not be overloaded.
3 days later, the Imperial College study made the same point in a much better and more rigorous way (among lots of other good points), but it was less widely shared on social media.
I’m not sure what the conclusion here is. Non-experts will sometimes make false assumptions and get the details wrong, but they’re still capable of making good points that only require you to multiply numbers together, and will do so a few days faster and in a way that’s more memetically fit than papers from experts?
Can you link to Nostalgebrist saying that his post was a math error at SSC? I can’t find it. Also, see Nostalgebrist’s update at the start of the critique:
To be clear, Bach’s use of a Gaussian is not the core problem here, it’s just a symptom of the core problem.
The core problem is that his curves do not come from a model of how disease is acquired, transmitted, etc. Instead they are a convenient functional form fitted to some parameters, with Bach making the call about which parameters should change – and how much – across different hypothetical scenarios.
Having a model is crucial when comparing one scenario to another, because it “keeps your accounting honest”: if you change one thing, everything causally downstream from that thing should also change.
Without a model, it’s possible to “forget” and not update a value after you change one of the inputs to that value.
That is what Bach does here: He assumes the number of total cases over the course of the epidemic will stay the same, whether or not we do what he calls “mild mitigation measures.” But the estimate he uses for this total – like most if not all such estimates out there – was computed directly from a specific value of the replication rate of the disease. Yet, all of the “mild mitigation measures” on the table right now would lower the replication rate of the disease – that’s what “slowing it down” means – and thus would lower the total.
Nowhere in that comment does he say that his post was or contained a “math error”. The closest thing I can find is this:
Great point about the step function. That convinces me that Bach would not have drawn a different qualitative conclusion if he had used a different functional form, no matter which one. I’ve updated my post with a note about this.
[EDIT: AFAICT Douglas_Knight is saying that Nostalgebrist’s initial guess that Bach’s post was sensitive to the functional form is a “math error”. I wouldn’t call it that, but perhaps reasonable people could disagree about this.]
What did his post originally mean? I’m not allowed to read people’s minds. He admits that no one took from it what he wanted them to take from it. Lanrian said that it was “a reasonable critique...that it doesn’t make sense to assume a normal distribution.” That was a qualitative complaint and he admitted that it was qualitatively wrong.
As I said, I do agree that the piece’s qualitative conclusion was correct. However, the gaussian assumption does make a large quantitative difference. Comparing it to the extreme: If we always have the maximum number of people in ICUs, continuously, the time until herd-immunity would be 4.9 years, which is a factor 3 less than what the normal assumption gives you. Although it is still clearly too much, it’s only one or two additional factors of 3 away from being reasonable. This extreme isn’t even that unrealistic; something like it could plausibly be achieved if the government continuously alternated between more or less lock-down, keeping the average R close to 1.
To be clear, I think that it’s good that the post was written, but I think it would have been substantially better if it had used a constant number of infected people. If you’re going to use unrealistic models (which, in many cases, you should!) it’s good practice to use the most conservative model possible, to ensure that your conclusion holds no matter what, and to let your reader see that it holds no matter what. In addition, it would have been simpler (you just have to multiply/divide 3 numbers), and it would have looked less like realistic-and-authoritative-math to the average reader, which would have better communicated its real epistemic status.
Notalgebraist posted a reasonable critique of Flattening The Curve Is a Deadly Delusion, explaining how it’s incorrect to assume that flattenings won’t reduce the total number of cases, and that it doesn’t make sense to assume a normal distribution.
I see the Deadly Delusion post saying “My back-of-the-envelope calculation is not a proper simulation, or a good model of what’s going on either. Don’t cite it as such! In reality, the spread of a disease does not follow a normal distribution. The main bump of the curve will be on the left, with a long tail on the right. There is always going to be some effective mitigation (prevention of public gatherings, conferences, non-essential travel). The model is quite sensitive to the length of the stay in the ICU. If we get that down, fewer people will need these resources simultaneously, and the peaks of the curves will come down. We may be able to fight the inflammation during pneumonia, and reduce the number of critical cases. The available medical resources will increase over time to deal with the need. Regulations will be dropped, new treatments will be explored, and some of them will work. At some point in the near future, we may have to blow into a tube before we enter an airplane or an important public building, and a little screen tells us within seconds if our airways hold COVID-19, H1N1 or the common flu. But the point of my argument is not that we are doomed, or that 6% of our population has to die, but that we must understand that containment is unavoidable, and should not be postponed, because later containment is going to be less effective and more expensive, and leads to additional deaths.” This seems to address Notalgebraist’s concerns pretty well? I thought maybe it was added to the post in response to feedback, but I see it in the first wayback capture https://web.archive.org/web/20200314031533/https://medium.com/@joschabach/flattening-the-curve-is-a-deadly-delusion-eea324fe9727 which looks like it’s older than https://nostalgebraist.tumblr.com/post/612592471097147392/flattening-the-curve-is-a-deadly-delusion
3 days later, the Imperial College study made the same point in a much better and more rigorous way (among lots of other good points), but it was less widely shared on social media.
I actually do not think I’ve seen any honest recounting of the Imperial College study. Unless I’m misreading it, the study appears to be saying that we either need to have essentially a complete lock down for the next 18 months (which **definitely has not** been reported by most media organizations) or we shouldn’t be implementing full social distancing, but rather an optimal policy of social distancing of those over 70 and school closures. Complete social distancing followed by relaxation before the development of a vaccine leads to almost as many deaths as unmitigated spread.
I’ve seen so much discussion of how the Imperial College paper has influenced governmental policy, but our current approach (which seems to be—hard, social isolation for a month or two followed by relaxation) is exactly the approach proscribed by the paper.
I would love if someone could weigh in with how I’m misinterpreting the paper because the distance between what the paper seems to say and the rhetoric I’ve seen online seems to be incredibly large. For instance, if I am interpreting the paper correctly, China is not a success case but will soon start seeing exponential growth again once they re-open unless they implement SDOL_70.
The excess deaths in the model are caused by a red line of critical care capacity, which the study assumes is fixed at 14 per 100,000 population in the US. If the curve of cases requiring critical care rises too high, 100% of cases above the red line will die even though 50% could have been saved.
But the coronavirus doesn’t need the entire package of critical care a hospital might provide, just a ventilator and a bed. So the US is aiming to simply build enough ventilators to hike the red line above the curve. (I would assume the UK is doing the same thing but I haven’t been following their response.)
For countries able to achieve it, this leaves suppression as the preferred policy option. [...] The major challenge of suppression is that this type of intensive intervention package—or something equivalently effective at reducing transmission—will need to be maintained until a vaccine becomes available (potentially 18 months or more)--given that we predict that transmission will quickly rebound if interventions are relaxed. We show that intermittent social distancing—triggered by trends in disease surveillance—may allow interventions to be relaxed temporarily in relative short time windows, but measures will need to be reintroduced if or when case numbers rebound.
So it’s not really saying “lock down for 18 months or do nothing”. It’s saying lock down until the problem is controlled and then squash new outbreaks quickly. The Hammer and the Dance article makes this point more clearly in my opinion, especially pointing out that a temporary lockdown would give us time to build up test and treatment capacity, protective equipment supplies, etc, and implement strategies for tracing and suppressing the new cases that arise after it’s lifted. However it does say “a few weeks” of strict lockdown will be enough without really supporting it well, and that seems optimistic (Bill Gates, for instance, has been saying 6-10 weeks).
Hm. Doesn’t the paper go on to say that full lockdowns would need to be in effect for 2⁄3 of those 18 months? I will read that article, but I don’t think the paper is saying that we could return to normal with monitoring and case tracing and localized lockdown to quash outbreaks for the remainder of the 18 months.
e: Okay, read the paper. Respectfully, many of the estimated numbers there seem entirely inconsistent with the literature that I’ve been reading from epidemiology experts. I haven’t seen a single paper estimating 10 million deaths in the United States, and I’m not inclined to trust an uncredentialed medium post (I know this is relevant to the OP topic).
I also really value modeling. Ferguson ran the numbers and seems to suggest that “the dance” would not be an effective approach for suppression long term and that we would need to go under frequent shelter-in-places again. This medium article doesn’t seem to cite any sort of number crunching that the hammer followed by the dance would work for long-term suppression.
It said the fraction would be a bit lower for the US because local outbreaks could be dealt with by state-level lockdowns, but I didn’t see a hard number. Still, intermittent lockdowns for 18 months seems much more achievable than a continuous lockdown. The Hammer article is definitely more optimistic than the Imperial paper, though it still doesn’t quite imply “return to normal”.
Since “do nothing” is not a real option, what will actually happen in the US (I’m moderately confident) is some degree of lockdown for several weeks to several months depending on how effective it is. The sooner we start it the better it will work. After that we will either: 1) give up, if the lockdown was ineffective and a large fraction of the country is infected (this is “flattening”), or 2) if the lockdown succeeded in reducing the number of cases substantially, we’ll move into a period of intermittent and possibly localized lockdowns interspersed with trying to test and contact trace. The fraction of time we spend in intermittent lockdowns will depend on how effective the testing and tracing is.
“Prescribe means “to set down authoritatively for direction” or “to set down a medical procedure in order to cure or alleviate symptoms.” The noun form is prescription, that is, something prescribed. Proscribe means “prohibit or limit” or “ostracize or avoid in a social sense”
Notalgebraist posted a reasonable critique of Flattening The Curve Is a Deadly Delusion, explaining how it’s incorrect to assume that flattenings won’t reduce the total number of cases, and that it doesn’t make sense to assume a normal distribution. I think that the the piece’s main point was correct, though: that the curve would need to get crazy flat for hospitals to not be overloaded.
3 days later, the Imperial College study made the same point in a much better and more rigorous way (among lots of other good points), but it was less widely shared on social media.
I’m not sure what the conclusion here is. Non-experts will sometimes make false assumptions and get the details wrong, but they’re still capable of making good points that only require you to multiply numbers together, and will do so a few days faster and in a way that’s more memetically fit than papers from experts?
The gaussian assumption makes no difference. Nostalgebrist’s post is a math error. He later admitted that at SSC, but barely updated his post.
Can you link to Nostalgebrist saying that his post was a math error at SSC? I can’t find it. Also, see Nostalgebrist’s update at the start of the critique:
here
Nowhere in that comment does he say that his post was or contained a “math error”. The closest thing I can find is this:
[EDIT: AFAICT Douglas_Knight is saying that Nostalgebrist’s initial guess that Bach’s post was sensitive to the functional form is a “math error”. I wouldn’t call it that, but perhaps reasonable people could disagree about this.]
You’ve lost track of the object level here.
What did his post originally mean? I’m not allowed to read people’s minds. He admits that no one took from it what he wanted them to take from it. Lanrian said that it was “a reasonable critique...that it doesn’t make sense to assume a normal distribution.” That was a qualitative complaint and he admitted that it was qualitatively wrong.
As I said, I do agree that the piece’s qualitative conclusion was correct. However, the gaussian assumption does make a large quantitative difference. Comparing it to the extreme: If we always have the maximum number of people in ICUs, continuously, the time until herd-immunity would be 4.9 years, which is a factor 3 less than what the normal assumption gives you. Although it is still clearly too much, it’s only one or two additional factors of 3 away from being reasonable. This extreme isn’t even that unrealistic; something like it could plausibly be achieved if the government continuously alternated between more or less lock-down, keeping the average R close to 1.
To be clear, I think that it’s good that the post was written, but I think it would have been substantially better if it had used a constant number of infected people. If you’re going to use unrealistic models (which, in many cases, you should!) it’s good practice to use the most conservative model possible, to ensure that your conclusion holds no matter what, and to let your reader see that it holds no matter what. In addition, it would have been simpler (you just have to multiply/divide 3 numbers), and it would have looked less like realistic-and-authoritative-math to the average reader, which would have better communicated its real epistemic status.
I see the Deadly Delusion post saying “My back-of-the-envelope calculation is not a proper simulation, or a good model of what’s going on either. Don’t cite it as such! In reality, the spread of a disease does not follow a normal distribution. The main bump of the curve will be on the left, with a long tail on the right. There is always going to be some effective mitigation (prevention of public gatherings, conferences, non-essential travel). The model is quite sensitive to the length of the stay in the ICU. If we get that down, fewer people will need these resources simultaneously, and the peaks of the curves will come down. We may be able to fight the inflammation during pneumonia, and reduce the number of critical cases. The available medical resources will increase over time to deal with the need. Regulations will be dropped, new treatments will be explored, and some of them will work. At some point in the near future, we may have to blow into a tube before we enter an airplane or an important public building, and a little screen tells us within seconds if our airways hold COVID-19, H1N1 or the common flu. But the point of my argument is not that we are doomed, or that 6% of our population has to die, but that we must understand that containment is unavoidable, and should not be postponed, because later containment is going to be less effective and more expensive, and leads to additional deaths.” This seems to address Notalgebraist’s concerns pretty well? I thought maybe it was added to the post in response to feedback, but I see it in the first wayback capture https://web.archive.org/web/20200314031533/https://medium.com/@joschabach/flattening-the-curve-is-a-deadly-delusion-eea324fe9727 which looks like it’s older than https://nostalgebraist.tumblr.com/post/612592471097147392/flattening-the-curve-is-a-deadly-delusion
While the paper itself wasn’t widely shared (not too surprising!) lots of news stories that cited it and passed on its conclusions were shared: https://www.nytimes.com/2020/03/17/world/europe/coronavirus-imperial-college-johnson.html https://www.washingtonpost.com/world/europe/a-chilling-scientific-paper-helped-upend-us-and-uk-coronavirus-strategies/2020/03/17/aaa84116-6851-11ea-b199-3a9799c54512_story.html https://www.cnn.com/2020/03/17/health/coronavirus-uk-model-study/index.html
I actually do not think I’ve seen any honest recounting of the Imperial College study. Unless I’m misreading it, the study appears to be saying that we either need to have essentially a complete lock down for the next 18 months (which **definitely has not** been reported by most media organizations) or we shouldn’t be implementing full social distancing, but rather an optimal policy of social distancing of those over 70 and school closures. Complete social distancing followed by relaxation before the development of a vaccine leads to almost as many deaths as unmitigated spread.
I’ve seen so much discussion of how the Imperial College paper has influenced governmental policy, but our current approach (which seems to be—hard, social isolation for a month or two followed by relaxation) is exactly the approach proscribed by the paper.
I would love if someone could weigh in with how I’m misinterpreting the paper because the distance between what the paper seems to say and the rhetoric I’ve seen online seems to be incredibly large. For instance, if I am interpreting the paper correctly, China is not a success case but will soon start seeing exponential growth again once they re-open unless they implement SDOL_70.
The excess deaths in the model are caused by a red line of critical care capacity, which the study assumes is fixed at 14 per 100,000 population in the US. If the curve of cases requiring critical care rises too high, 100% of cases above the red line will die even though 50% could have been saved.
But the coronavirus doesn’t need the entire package of critical care a hospital might provide, just a ventilator and a bed. So the US is aiming to simply build enough ventilators to hike the red line above the curve. (I would assume the UK is doing the same thing but I haven’t been following their response.)
From the Imperial College paper:
So it’s not really saying “lock down for 18 months or do nothing”. It’s saying lock down until the problem is controlled and then squash new outbreaks quickly. The Hammer and the Dance article makes this point more clearly in my opinion, especially pointing out that a temporary lockdown would give us time to build up test and treatment capacity, protective equipment supplies, etc, and implement strategies for tracing and suppressing the new cases that arise after it’s lifted. However it does say “a few weeks” of strict lockdown will be enough without really supporting it well, and that seems optimistic (Bill Gates, for instance, has been saying 6-10 weeks).
Hm. Doesn’t the paper go on to say that full lockdowns would need to be in effect for 2⁄3 of those 18 months? I will read that article, but I don’t think the paper is saying that we could return to normal with monitoring and case tracing and localized lockdown to quash outbreaks for the remainder of the 18 months.
e: Okay, read the paper. Respectfully, many of the estimated numbers there seem entirely inconsistent with the literature that I’ve been reading from epidemiology experts. I haven’t seen a single paper estimating 10 million deaths in the United States, and I’m not inclined to trust an uncredentialed medium post (I know this is relevant to the OP topic).
I also really value modeling. Ferguson ran the numbers and seems to suggest that “the dance” would not be an effective approach for suppression long term and that we would need to go under frequent shelter-in-places again. This medium article doesn’t seem to cite any sort of number crunching that the hammer followed by the dance would work for long-term suppression.
It said the fraction would be a bit lower for the US because local outbreaks could be dealt with by state-level lockdowns, but I didn’t see a hard number. Still, intermittent lockdowns for 18 months seems much more achievable than a continuous lockdown. The Hammer article is definitely more optimistic than the Imperial paper, though it still doesn’t quite imply “return to normal”.
Since “do nothing” is not a real option, what will actually happen in the US (I’m moderately confident) is some degree of lockdown for several weeks to several months depending on how effective it is. The sooner we start it the better it will work. After that we will either: 1) give up, if the lockdown was ineffective and a large fraction of the country is infected (this is “flattening”), or 2) if the lockdown succeeded in reducing the number of cases substantially, we’ll move into a period of intermittent and possibly localized lockdowns interspersed with trying to test and contact trace. The fraction of time we spend in intermittent lockdowns will depend on how effective the testing and tracing is.
Proscribe or prescribe?
“Prescribe means “to set down authoritatively for direction” or “to set down a medical procedure in order to cure or alleviate symptoms.” The noun form is prescription, that is, something prescribed. Proscribe means “prohibit or limit” or “ostracize or avoid in a social sense”
Yep, I meant it in the correct fashion :) Prescribe would imply the opposite of what the paper said.