I agree that the first few chapters of Jaynes are illuminating, haven’t tried to read further. Bayesian Data Analysis by Gelman feels much more practical at least for what I personally need (a reference book for statistical techniques).
The general pre-requisites are actually spelled out in the introduction of Jayne’s Probability Theory. Emphasis mine.
The following material is addressed to readers who are already familiar with applied mathematics at the advanced undergraduate level or preferably higher; and with some field, such as physics, chemistry, biology, geology, medicine, economics, sociology, engineering, operations research, etc., where inference is needed. A previous acquaintance with probability and statistics is not necessary; indeed, a certain amount of innocence in this area may be desirable, because there will be less to unlearn.
Basic course on Stochastics and Probability theory via Kolmogorov’s approach, with perhaps some real analysis. If you want to keep reading upper level books then there’s no reason to stop at Jaynes, there’s differential geometry as applied in information geometry.
familiar with applied mathematics at the advanced undergraduate level or preferably higher
In working through the text, I have found that my undergraduate engineering degree and mathematics minor would not have been sufficient to understand the details of Jaynes’ arguments, following the derivations and solving the problems. I took some graduate courses in math and statistics, and more importantly I’ve picked up a smattering of many fields of math after my formal education, and these plus Google have sufficed.
Be advised that there are errors (typographical, mathematical, rhetorical) in the text that can be confusing if you try to follow Jaynes’ arguments exactly. Furthermore, it is most definitely written in a blustering manner (to bully his colleagues and others who learned frequentist statistics) rather than in an educational manner (to teach someone statistics for the first time). So if you want to use the text to learn the subject matter, I strongly recommend you take the denser parts slowly and invent problems based on them for yourself to solve.
I find it impossible not to constantly sense in Jaynes’ tone, and especially in his many digressions propounding his philosophies of various things, the same cantankerous old-man attitude that I encounter most often in cranks. The difference is that Jaynes is not a crackpot; whether by wisdom or luck, the subject matter that became his cranky obsession is exquisitely useful for remaining sane.
already familiar with applied mathematics … where inference is needed
That’s where Jaynes shines. Many mathematical subjects are treated axiomatically. Jaynes instead starts from the basic problem of representing uncertainty. Churning out the implications of axioms is a very different mindset than “I have data, what can I conclude from it?”
I think this is true as well.
A previous acquaintance with probability and statistics is not necessary; indeed, a certain amount of innocence in this area may be desirable, because there will be less to unlearn.
See this discussion in The Best Textbooks on Every Subject
I agree that the first few chapters of Jaynes are illuminating, haven’t tried to read further. Bayesian Data Analysis by Gelman feels much more practical at least for what I personally need (a reference book for statistical techniques).
The general pre-requisites are actually spelled out in the introduction of Jayne’s Probability Theory. Emphasis mine.
I don’t know what that means. Calculus? Analysis? Linear algebra? Matrices? Non-euclidean geometry?
Basic course on Stochastics and Probability theory via Kolmogorov’s approach, with perhaps some real analysis. If you want to keep reading upper level books then there’s no reason to stop at Jaynes, there’s differential geometry as applied in information geometry.
In working through the text, I have found that my undergraduate engineering degree and mathematics minor would not have been sufficient to understand the details of Jaynes’ arguments, following the derivations and solving the problems. I took some graduate courses in math and statistics, and more importantly I’ve picked up a smattering of many fields of math after my formal education, and these plus Google have sufficed.
Be advised that there are errors (typographical, mathematical, rhetorical) in the text that can be confusing if you try to follow Jaynes’ arguments exactly. Furthermore, it is most definitely written in a blustering manner (to bully his colleagues and others who learned frequentist statistics) rather than in an educational manner (to teach someone statistics for the first time). So if you want to use the text to learn the subject matter, I strongly recommend you take the denser parts slowly and invent problems based on them for yourself to solve.
I find it impossible not to constantly sense in Jaynes’ tone, and especially in his many digressions propounding his philosophies of various things, the same cantankerous old-man attitude that I encounter most often in cranks. The difference is that Jaynes is not a crackpot; whether by wisdom or luck, the subject matter that became his cranky obsession is exquisitely useful for remaining sane.
Good quote.
But I would have bolded
That’s where Jaynes shines. Many mathematical subjects are treated axiomatically. Jaynes instead starts from the basic problem of representing uncertainty. Churning out the implications of axioms is a very different mindset than “I have data, what can I conclude from it?”
I think this is true as well.