I was curious about Zvi’s performance since he started making predictions about the US in mid-November, so I tossed together a (very quick and sloppy, inconsistently out of date due to some numbers being more updated) visual comparison:
Predictions are a week in advance. Dotted lines / triangles are predictions; blue is % tests positive, yellow is millions of tests administered, red is official deaths.
Crunching some numbers in a copy of the spreadsheet… Zvi’s predictions are better than the naive model of assuming next week’s numbers will be the same as this week’s numbers.
Biggest improvement over the null model for predicting deaths (mean squared error is 47% as big), smallest improvement for positive test % (MSE 80% as big), in between for number of tests (MSE 67% as big).
Although if I instead look at the predicted weekly change and compare it to the actual change that week, all three sets of predictions are roughly equally accurate with correlations (predicted change vs. actual change) between .52 and .58.
I was curious about Zvi’s performance since he started making predictions about the US in mid-November, so I tossed together a (very quick and sloppy, inconsistently out of date due to some numbers being more updated) visual comparison:
Predictions are a week in advance. Dotted lines / triangles are predictions; blue is % tests positive, yellow is millions of tests administered, red is official deaths.
Crunching some numbers in a copy of the spreadsheet… Zvi’s predictions are better than the naive model of assuming next week’s numbers will be the same as this week’s numbers.
Biggest improvement over the null model for predicting deaths (mean squared error is 47% as big), smallest improvement for positive test % (MSE 80% as big), in between for number of tests (MSE 67% as big).
Although if I instead look at the predicted weekly change and compare it to the actual change that week, all three sets of predictions are roughly equally accurate with correlations (predicted change vs. actual change) between .52 and .58.