Why: Sivia’s book is well suited for smart people who have not had little or no statistical training. It starts from the basics and covers a lot of important ground. I think it takes the right approach, first doing some simple examples where analytical solutions are available or it is feasible to integrate naively and numerically. Then it teaches into maximum likelihood estimation (MLE), how to do it and why it makes sense from a Bayesian perspective. I think MLE is a very very useful technique, especially so for engineers. I would overall recommend just Part I: The Essentials, I don’t think the second half is so useful, except perhaps the MLE extensions chapter. There are better places to learn about MCMC approximation.
Why not other books?
Bayesian Data Analysis by Gelman—Geared more for people who have done statistics before.
Bayesian Statistics by Bolstad—Doesn’t cover as much as Sivia’s book, most notably doesn’t cover MLE. Goes kinda slowly and spends a lot of time on comparing Bayesian statistics to Frequentist statistics.
The Bayesian Choice—more of a mathematical statistics book, not suited for beginners.
Brandon Reinhart used both Sivia’s book and Bolstad’s book and found (3rd message) Bolstad’s book better for those with no stats experience:
For statistics, I recommend An Introduction to Bayesian Statistics by William Bolstad. This is superior to the “Data Analysis” book if you’re learning stats from scratch. Both “Data Analysis” and “Bayesian Data Analysis” assume a certain base level of familiarity with the material. The Bolstad book will bootstrap you from almost no familiarity with stats through fairly clear explanations and good supporting exercises.
Nonetheless, it’s something you should do with other people. You may not notice what you aren’t completely comprehending otherwise. Do the exercises!
Based on these comments, I think I was underestimating inferential distance, and I now change my recommendation. You should read Bolstad’s book first (skipping the parts comparing bayesian and frequentist methods unless that’s important to you) and then read Sivia’s book. If you have experience with statistics you may start with Sivia’s book.
I don’t have an especially awesome place, but Bayesian Data Analysis by Gelman introduces the basics of Metropolis Hastings and Gibbs Sampling (those are probably the first ones to learn). There are probably quite a few other places to learn about these two algorithms too (including wikipedia). MCMC using Hamiltonian Dynamics by Neal, is the standard reference for Hamiltonian Monte Carlo (what I would suggest learning after those two).
Is Gleman’s book a good recommendation for people who have done frequentist statistics and/or combinatorics? I have free access to it and basic familiarity with both.
Update see my comment for new thoughts
Topic: Introductory Bayesian Statistics (as distinct from more advanced Bayesian statistics)
Recommendation: Data Analysis: A Bayesian Tutorial by Skilling and Sivia
Why: Sivia’s book is well suited for smart people who have not had little or no statistical training. It starts from the basics and covers a lot of important ground. I think it takes the right approach, first doing some simple examples where analytical solutions are available or it is feasible to integrate naively and numerically. Then it teaches into maximum likelihood estimation (MLE), how to do it and why it makes sense from a Bayesian perspective. I think MLE is a very very useful technique, especially so for engineers. I would overall recommend just Part I: The Essentials, I don’t think the second half is so useful, except perhaps the MLE extensions chapter. There are better places to learn about MCMC approximation.
Why not other books?
Bayesian Data Analysis by Gelman—Geared more for people who have done statistics before.
Bayesian Statistics by Bolstad—Doesn’t cover as much as Sivia’s book, most notably doesn’t cover MLE. Goes kinda slowly and spends a lot of time on comparing Bayesian statistics to Frequentist statistics.
The Bayesian Choice—more of a mathematical statistics book, not suited for beginners.
Brandon Reinhart used both Sivia’s book and Bolstad’s book and found (3rd message) Bolstad’s book better for those with no stats experience:
Based on these comments, I think I was underestimating inferential distance, and I now change my recommendation. You should read Bolstad’s book first (skipping the parts comparing bayesian and frequentist methods unless that’s important to you) and then read Sivia’s book. If you have experience with statistics you may start with Sivia’s book.
Nice!
See also https://stats.stackexchange.com/questions/125/what-is-the-best-introductory-bayesian-statistics-textbook.
Any in particular? I came to this thread seeking exactly this.
I don’t have an especially awesome place, but Bayesian Data Analysis by Gelman introduces the basics of Metropolis Hastings and Gibbs Sampling (those are probably the first ones to learn). There are probably quite a few other places to learn about these two algorithms too (including wikipedia). MCMC using Hamiltonian Dynamics by Neal, is the standard reference for Hamiltonian Monte Carlo (what I would suggest learning after those two).
Is Gleman’s book a good recommendation for people who have done frequentist statistics and/or combinatorics? I have free access to it and basic familiarity with both.