It’s common for much simpler Statistical Prediction Rules, such as linear regression or even simpler models, to outperform experts even when they were built to predict the experts’ judgment.
Yeah, ‘transcendence’ is a truly overhyped and absurd way to label what appears to be a simple ensemble effect. Should Galton have said that the crowd ‘transcended’ in guessing the weight of a fat pig?
This isn’t even a useful motivating example for how LLMs could outperform human experts while doing offline learning, because the averaging benefit breaks down, as expected, at their high end.
It’s common for much simpler Statistical Prediction Rules, such as linear regression or even simpler models, to outperform experts even when they were built to predict the experts’ judgment.
Yeah, ‘transcendence’ is a truly overhyped and absurd way to label what appears to be a simple ensemble effect. Should Galton have said that the crowd ‘transcended’ in guessing the weight of a fat pig?
This isn’t even a useful motivating example for how LLMs could outperform human experts while doing offline learning, because the averaging benefit breaks down, as expected, at their high end.
Exactly. All that’s needed for “transcendence” is removing some noise.
I highly recommend the book Noise by Daniel Kahneman et al on this topic.