Problem is, “throw a massive neural network at it” fails completely for the vast majority of practical applications. We need astronomical amounts of data to make neural networks work. Try using them at a small company on a problem with a few thousand data points; it won’t work.
I see the moral of that story as: if you have enough data, any stupid algorithm will work. It’s when data is not superabundant that we need Bayesian methods, because nothing else reliably works. (BTW, this is something we could guess based on Bayesian foundations: Cox’ Theorem or an information-theoretic foundation of Bayesian probability do not depend on infinite data for any particular problem, whereas things like frequentist statistics or brute-force neural nets do.)
(Side note for people confused about how that plays with the comment at the top of this thread: the relevant limit there was not the limit of infinite data, but the limit of reasoning over all possible models.)
I claim that the way bayesianism has been presented around here (as an ideal of rationality) is not a falsifiable framework, and so at the very least we need someone else to make the case for what they’re standing for.
Around here, rationality is about winning. To the extent that we consider Bayesianism an ideal of rationality, that can be falsified by outperforming Bayesianism, in places where behavior of that ideal can be calculated or at least characterized enough to prove that something else outperforms the supposed ideal.
Problem is, “throw a massive neural network at it” fails completely for the vast majority of practical applications. We need astronomical amounts of data to make neural networks work. Try using them at a small company on a problem with a few thousand data points; it won’t work.
I see the moral of that story as: if you have enough data, any stupid algorithm will work. It’s when data is not superabundant that we need Bayesian methods, because nothing else reliably works. (BTW, this is something we could guess based on Bayesian foundations: Cox’ Theorem or an information-theoretic foundation of Bayesian probability do not depend on infinite data for any particular problem, whereas things like frequentist statistics or brute-force neural nets do.)
(Side note for people confused about how that plays with the comment at the top of this thread: the relevant limit there was not the limit of infinite data, but the limit of reasoning over all possible models.)
Around here, rationality is about winning. To the extent that we consider Bayesianism an ideal of rationality, that can be falsified by outperforming Bayesianism, in places where behavior of that ideal can be calculated or at least characterized enough to prove that something else outperforms the supposed ideal.