I mostly made my comment to point out that the particular question that’s being used as evidence for expert incompetence may have been unusually difficult to get right. So I don’t want to appear as though I’m confidently claiming that experts need a lesson on forecasting.
That said, I think some people would indeed become a bit better calibrated and we’d see wider confidence intervals from them in the future.
I think the main people who would do well to join Metaculus are people like Ioannidis or the Oxford CEBM people who sling out these unreasonably low IFR estimates. If you’re predicting all kinds of things about this virus 24⁄7 you’ll realize eventually that reality is not consistent with “this is at most mildly worse than the flu.”
What would you expect to happen if those experts started participating in Metaculus?
I mostly made my comment to point out that the particular question that’s being used as evidence for expert incompetence may have been unusually difficult to get right. So I don’t want to appear as though I’m confidently claiming that experts need a lesson on forecasting.
That said, I think some people would indeed become a bit better calibrated and we’d see wider confidence intervals from them in the future.
I think the main people who would do well to join Metaculus are people like Ioannidis or the Oxford CEBM people who sling out these unreasonably low IFR estimates. If you’re predicting all kinds of things about this virus 24⁄7 you’ll realize eventually that reality is not consistent with “this is at most mildly worse than the flu.”