“I thought the whole point of probabilistic methods is that it doesn’t matter too much what the prior is, it will always eventually converge on the right answer...”
AIUI this is somewhat misleading. Bayesian methods are most valuable precisely when the amount of available data is limited and prior probability is important. Whenever “it doesn’t matter too much what the prior is”, it makes more sense to use frequentist methods, which rely on large amounts of data to converge to the right solution.
Of course frequentist tools also make assumptions about the data and some of these assumptions may be disguised and poorly understood (making sense of these is arguably part of the “searching for Bayes structure” program), but some interpretations are straightforward: for instance, likelihood-based methods are equivalent to Bayesian methods assuming a uniform prior distribution.
(As an aside, it’s ironic that Bayesian interpretation of such statistical tools is being pursued for the sake of rigor, given that frequentist statistics itself was developed as a reaction to widespread ad-hoc use of the “principle of inverse probability”.)
“I thought the whole point of probabilistic methods is that it doesn’t matter too much what the prior is, it will always eventually converge on the right answer...”
AIUI this is somewhat misleading. Bayesian methods are most valuable precisely when the amount of available data is limited and prior probability is important. Whenever “it doesn’t matter too much what the prior is”, it makes more sense to use frequentist methods, which rely on large amounts of data to converge to the right solution.
Of course frequentist tools also make assumptions about the data and some of these assumptions may be disguised and poorly understood (making sense of these is arguably part of the “searching for Bayes structure” program), but some interpretations are straightforward: for instance, likelihood-based methods are equivalent to Bayesian methods assuming a uniform prior distribution.
(As an aside, it’s ironic that Bayesian interpretation of such statistical tools is being pursued for the sake of rigor, given that frequentist statistics itself was developed as a reaction to widespread ad-hoc use of the “principle of inverse probability”.)