The problems with Bayes are suffcieintly non-obvious to have eluded many or most at LW.
On the one hand, I think that page in specific is actually based on outdated Bayesian methods, and there’s been a lot of good work in Bayesian statistics for complex models and cognitive science in recent years.
On the other hand, I freaking love that website, despite its weirdo Buddhist-philosophical leanings and one or two things it gets Wrong according to my personal high-and-mighty ideologies.
And on the gripping hand, he is very, very right that the way the LW community tends to phrase things in terms of “just Bayes it” is not only a mischaracterization of the wide world of statistics, it’s even an oversimplification of Bayesian statistics as a subfield. Bayes’ Law is just the update/training rule! You also need to discuss marginalization; predictive distributions; maximum-entropy priors, structural simplicity priors, and Bayesian Occam’s Razor, and how those are three different views of Occam’s Razor that have interesting similarities and differences; model selection; the use of Bayesian point-estimates and credible-hypothesis tests for decision-making; equivalent sample sizes; conjugate families; and computational Bayes methods.
Then you’re actually learning and doing Bayesian statistics.
On the miniature nongripping hand, I can’t help but feel that the link between probability, thermodynamics, and information theory means Eliezer and the Jaynesians are probably entirely correct that as a physical fact, real-world event frequencies and movements of information obey Bayes’ Law with respect to the information embodied in the underlying physics, whether or not I can model any of that well or calculate posterior distributions feasibly.
On the one hand, I think that page in specific is actually based on outdated Bayesian methods, and there’s been a lot of good work in Bayesian statistics for complex models and cognitive science in recent years.
On the other hand, I freaking love that website, despite its weirdo Buddhist-philosophical leanings and one or two things it gets Wrong according to my personal high-and-mighty ideologies.
And on the gripping hand, he is very, very right that the way the LW community tends to phrase things in terms of “just Bayes it” is not only a mischaracterization of the wide world of statistics, it’s even an oversimplification of Bayesian statistics as a subfield. Bayes’ Law is just the update/training rule! You also need to discuss marginalization; predictive distributions; maximum-entropy priors, structural simplicity priors, and Bayesian Occam’s Razor, and how those are three different views of Occam’s Razor that have interesting similarities and differences; model selection; the use of Bayesian point-estimates and credible-hypothesis tests for decision-making; equivalent sample sizes; conjugate families; and computational Bayes methods.
Then you’re actually learning and doing Bayesian statistics.
On the miniature nongripping hand, I can’t help but feel that the link between probability, thermodynamics, and information theory means Eliezer and the Jaynesians are probably entirely correct that as a physical fact, real-world event frequencies and movements of information obey Bayes’ Law with respect to the information embodied in the underlying physics, whether or not I can model any of that well or calculate posterior distributions feasibly.