In Bayesian statistics, Gelman’s Bayesian Data Analysis, 2nd ed (I hear a third edition is coming soon) instead of Jaynes’s Probability Theory: The Logic of Science (but do read the first two chapters of Jaynes) and Bernardo’s Bayesian Theory.
I just made the same recommendation on a different post. My reasons for recommending Gelman over Jaynes here is the practical value of working through the problems in Gelman’s book. The problems Jaynes gives are focused on the theoretical, but the problems in BDA are applied, computational, and this is true from the beginning of the book: I used R for many of the problems at the end of chapter 2. By the end of chapter 3 I could already see ways I could apply the things I learned from BDA to my work as a Data Scientist. Jaynes also gives far fewer exercises—there are maybe 20-30 in the whole book, but in BDA there are 15 or so per chapter so far.
I read Jaynes’s book cover-to-cover, but should confess I’m only through chapter 3 of BDA. So maybe it goes off the deep end and I come back here in 6 months and withdraw my recommendation. But right now I’m recommending Bayesian Data Analysis.
Gelman’s text is very specifically targeted at the kinds of problems he enjoys in sociology and politics, though. If you’re interested in solving problems in that field or like it (highly complex unobservable mechanisms, large number of potential causes and covariates, sensible multiple groupings of observations, etc) then his book is great. If you’re looking at problems more like in physics, then it won’t help you at all and you’re better off reading Jaynes’.
(Also recommended over Gelman’s Applied Regression and Modeling if the above condition holds.)
Ah, interesting. I used the material I learned from that book in my thesis on data analysis for proteomics, so you can expand the list of topics to include biological data too; biology problems tend to fit your list of problem characteristics.
highly complex unobservable mechanisms, large number of potential causes and covariates, sensible multiple groupings of observations, etc
Hmm, I might be totally off base here, but wouldn’t that sort of thing be useful for reasoning about highly powerful optimization processes that would be driven to maximize their expected utility by figuring out what actions would decrease the entropy of a desirable portion of state space by working from massive amounts of input data? Maybe I should check it out either way.
I’m sorry, as I’m reading it that sounds rather vague. Gelman’s work stems largely from the fact that there is no central theory of political action. Group behavior is some kind of sum of individual behaviors, but with only aggregate measurements you cannot discern the individual causes. This leads to a tendency to never see zero effect sizes, for instance.
In Bayesian statistics, Gelman’s Bayesian Data Analysis, 2nd ed (I hear a third edition is coming soon) instead of Jaynes’s Probability Theory: The Logic of Science (but do read the first two chapters of Jaynes) and Bernardo’s Bayesian Theory.
I just made the same recommendation on a different post. My reasons for recommending Gelman over Jaynes here is the practical value of working through the problems in Gelman’s book. The problems Jaynes gives are focused on the theoretical, but the problems in BDA are applied, computational, and this is true from the beginning of the book: I used R for many of the problems at the end of chapter 2. By the end of chapter 3 I could already see ways I could apply the things I learned from BDA to my work as a Data Scientist. Jaynes also gives far fewer exercises—there are maybe 20-30 in the whole book, but in BDA there are 15 or so per chapter so far.
I read Jaynes’s book cover-to-cover, but should confess I’m only through chapter 3 of BDA. So maybe it goes off the deep end and I come back here in 6 months and withdraw my recommendation. But right now I’m recommending Bayesian Data Analysis.
Cyan,
Could you give us some reasons?
Both Jaynes’s and Bernardo’s texts have a lot of material on why one ought to do Bayesian statistics; Gelman text excels in showing how to do it.
Gelman’s text is very specifically targeted at the kinds of problems he enjoys in sociology and politics, though. If you’re interested in solving problems in that field or like it (highly complex unobservable mechanisms, large number of potential causes and covariates, sensible multiple groupings of observations, etc) then his book is great. If you’re looking at problems more like in physics, then it won’t help you at all and you’re better off reading Jaynes’.
(Also recommended over Gelman’s Applied Regression and Modeling if the above condition holds.)
Ah, interesting. I used the material I learned from that book in my thesis on data analysis for proteomics, so you can expand the list of topics to include biological data too; biology problems tend to fit your list of problem characteristics.
Hmm, I might be totally off base here, but wouldn’t that sort of thing be useful for reasoning about highly powerful optimization processes that would be driven to maximize their expected utility by figuring out what actions would decrease the entropy of a desirable portion of state space by working from massive amounts of input data? Maybe I should check it out either way.
I’m sorry, as I’m reading it that sounds rather vague. Gelman’s work stems largely from the fact that there is no central theory of political action. Group behavior is some kind of sum of individual behaviors, but with only aggregate measurements you cannot discern the individual causes. This leads to a tendency to never see zero effect sizes, for instance.
Thanks. I added this to the list.