Quant, systems thinker, anarchist.
I write at https://entropicthoughts.com
My inbox is lw[at]xkqr.org
kqr
Karma: 269
The AI forecaster is able to consistently outperform the crowd forecast on a sufficiently large number of randomly selected questions on a high-quality forecasting platform
Seeing how the crowd forecast routinely performs at a superhuman level itself, isn’t it an unfairly high bar to clear? Not invalidating the rest of your arguments – the methodological problems you point out are really bad – but before asking the question about superhuman performance it makes a lot of sense to fully agree on what superhuman performance really is.
(I also note that a high-quality forecasting platform suffers from self-selection by unusually enthusiastic forecasters, bringing up the bar further. However, I don’t believe this to be an actual problem because if someone is claiming “performance on par with humans” I would expect that to mean “enthusiastic humans”.)
Thanks everyone. I had a great time!