Academics are indeed very smart, but under time pressure they have many additional constraints, most particularly the need to have everything pass peer review (now or later), which entails some unfortunate requirements like
“tighter confidence intervals look better”
“a single model is more justifiable than an ensemble”
“you can justify a handpicked parameter more easily than a handpicked distribution over that parameter”
“if your model looks at all like someone else’s you’d better cite them, so either keep things in spherical cow territory or do a long literature search while people are dying”
We’re not constrained by the same factors, and so it’s perhaps possible to do better.
I have no doubt that LW is more than capable of making models beyond *my* ability to find fault with.
And I am actually confused that “Our models won’t pass peer review” is being used as evidence of higher quality.
Is there a betting market where I can take the house position against modelers who think they can outperform some publicly available professional epidemiologist’s model?
Nobody’s forcing you to help with this! And if you just want to point out why particular proposed models are bad, that’s a good way to help as well.
Academics are indeed very smart, but under time pressure they have many additional constraints, most particularly the need to have everything pass peer review (now or later), which entails some unfortunate requirements like
“tighter confidence intervals look better”
“a single model is more justifiable than an ensemble”
“you can justify a handpicked parameter more easily than a handpicked distribution over that parameter”
“if your model looks at all like someone else’s you’d better cite them, so either keep things in spherical cow territory or do a long literature search while people are dying”
We’re not constrained by the same factors, and so it’s perhaps possible to do better.
I have no doubt that LW is more than capable of making models beyond *my* ability to find fault with.
And I am actually confused that “Our models won’t pass peer review” is being used as evidence of higher quality.
Is there a betting market where I can take the house position against modelers who think they can outperform some publicly available professional epidemiologist’s model?