How about cancer research? This page lists rate of success of clinical trials in different subfields; oncology clinical trials have a success rate of around 4%. I would also guess that a large chunk of the “successes” in fact do basically-nothing and made it through largely by being the one-in-twenty which hit 95% significance by chance, or managed to p-hack, or the like. From an inside view, most cancer research I’ve seen indeed looks pretty unhelpful based on my understanding of biology and how-science-works in general (and this goes double for any cancer research “using machine learning”, which is a hot subfield).
More generally: we live in a high-dimensional world. Figuring out “which direction to search in” is usually a much more taut constraint than having the resources to search. Brute-force searching a high-dimensional space requires resources exponential in the dimension of the space.
Combine that with misaligned incentives for researchers, and our default expectation should usually be that finding the right researchers to fund is more of a constraint than resources.
How about cancer research? This page lists rate of success of clinical trials in different subfields; oncology clinical trials have a success rate of around 4%. I would also guess that a large chunk of the “successes” in fact do basically-nothing and made it through largely by being the one-in-twenty which hit 95% significance by chance, or managed to p-hack, or the like. From an inside view, most cancer research I’ve seen indeed looks pretty unhelpful based on my understanding of biology and how-science-works in general (and this goes double for any cancer research “using machine learning”, which is a hot subfield).
More generally: we live in a high-dimensional world. Figuring out “which direction to search in” is usually a much more taut constraint than having the resources to search. Brute-force searching a high-dimensional space requires resources exponential in the dimension of the space.
Combine that with misaligned incentives for researchers, and our default expectation should usually be that finding the right researchers to fund is more of a constraint than resources.