FHI paper published in Science: interventions against COVID-19

[SoerenMind]

This is a linkpost for https://​​science.sciencemag.org/​​lookup/​​doi/​​10.1126/​​science.abd9338

Summary: this is basically a Bayesian regression of the COVID-19 cases and deaths in 41 countries against the measures taken by those countries. It requires a rough model of how infections spread which means you have to be careful about choosing models, validate them by predicting the epidemic, and try to deal with confounding. The results support closing schools and universities, gatherings, and some types of face-to-face businesses.

It’s been great to work on this with a large team of talented researchers from many universities, most of whom are rationalists and EAs. The lead authors and the senior author are employees and affiliates of FHI:

Jan M. Brauner*, Sören Mindermann*, Mrinank Sharma*, David Johnston, John Salvatier, Tomáš Gavenčiak, Anna B. Stephenson, Gavin Leech, George Altman, Vladimir Mikulik, Alexander John Norman, Joshua Teperowski Monrad, Tamay Besiroglu, Hong Ge, Meghan A. Hartwick, Yee Whye Teh, Leonid Chindelevitch, Yarin Gal, Jan Kulveit

We’re also grateful to Tim Telleen-Lawton and BERI who funded and operationally supported the Epidemic Forecasting project which in turn incubated this project.

Paper: Inferring the Effectiveness of Government Interventions Against COVID-19

There’s also a closely related NeurIPS paper.

The story of this paper is easy to tell with its figures so I’ll briefly to that.

Data:

Main results:

Fig. 2 NPI effectiveness under default model settings.Posterior percentage reductions in R_t with median, 50% and 95% prediction intervals shown. Prediction intervals reflect many sources of uncertainty, including NPI effectiveness varying by country and uncertainty in epidemiological parameters. A negative 1% reduction refers to a 1% increase in R_t. “Schools and universities closed” shows the joint effect of closing both schools and universities in conjunction; the individual effect of closing just one will be smaller (see text). Cumulative effects are shown for hierarchical NPIs (gathering bans and business closures) i.e., the result for “Most nonessential businesses closed” shows the cumulative effect of two NPIs with separate parameters and symbols—closing some (high-risk) businesses, and additionally closing most remaining (non-high-risk, but nonessential) businesses given that some businesses are already closed.

Effect of combined interventions:

Fig. 3 Combined NPI effectiveness for the 15 most commonly implemented sets of NPIs in our data.Solid and shaded regions denote 50% and 95% Bayesian prediction intervals. (A) Predicted R_t after implementation of each set of NPIs, assuming R₀ = 3.3. (B) Maximum R₀ that can be reduced to R_t below 1 by common sets of NPIs. Readers can interactively explore the effects of all sets of NPIs, while setting R₀ and adjusting NPI effectiveness to local circumstances, with our online mitigation calculator (16).

Lots of sensitivity analyses:

Fig. 4 Median NPI effectiveness across the sensitivity analyses.(A) Median NPI effectiveness (reduction in R_t) when varying different components of the model or the data in 206 experimental conditions. Results are displayed as violin plots, using kernel density estimation to create the distributions. Inside the violins, the box plots show median and interquartile-range. The vertical lines mark 0%, 17.5%, and 35% (see text). (B to E) Categorized sensitivity analyses. (B) Sensitivity to model structure. Using only cases or only deaths as observations (2 experimental conditions; fig. S7); varying the model structure (3 conditions; fig. S8, left). (C) Sensitivity to data and preprocessing. Leaving out countries from the dataset (42 conditions; figs. S5 and S21); varying the threshold below which cases and deaths are masked (8 conditions; fig. S13); sensitivity to correcting for undocumented cases and to country-level differences in case ascertainment (2 conditions; fig. S6). (D) Sensitivity to epidemiological parameters. Jointly varying the means of the priors over the means of the generation interval, the infection-to-case-confirmation delay, and the infection-to-death delay (125 conditions; fig. S10); varying the prior over R₀ (4 conditions; fig. S11); varying the prior over NPI effect parameters (3 conditions; fig. S11); varying the prior over the degree to which NPI effects vary across countries (3 conditions; fig. S12). (E) Sensitivity to unobserved factors influencing R_t. Excluding observed NPIs one at a time (8 conditions; fig. S9); controlling for additional NPIs from a different dataset (6 conditions; fig. S9).

Structure of the main model:

Fig. 5 Model overview.Unshaded, white nodes are observed. We describe the diagram from bottom to top: The mean effect parameter of NPI i is α_i, and the country-specific effect parameter is α_i,c. On each day t, a country’s daily reproduction number R_t,c depends on the country’s basic reproduction number R_0,c and the active NPIs. The active NPIs are encoded by x_i,t,c, which is 1 if NPI i is active in country c at time t, and 0 otherwise. R_t,c is transformed into the daily growth rate g_t,c using the generation interval parameters, and subsequently is used to compute the new infections N(C)t,cNt,c(C) and N(D)t,cNt,c(D) that will subsequently become confirmed cases and deaths, respectively. Finally, the expected number of daily confirmed cases y(C)t,cyt,c(C) and deaths y(D)t,cyt,c(D) are computed using discrete convolutions of N(.)t,cNt,c(.) with the relevant delay distributions. Our model uses both case and death data: it splits all nodes above the daily growth rate g_t,c into separate branches for deaths and confirmed cases. We account for uncertainty in the generation interval, infection to case confirmation delay, and the infection to death delay by placing priors over the parameters of these distributions.

Much of the interesting parts are in the appendix.