I’ve been teaching myself the basics of probability theory (I’m sixteen) but I’m having trouble on the first step. My basic definitions of probabilities are all frequentist, and I don’t know a good Bayesian source appropriate for a secondary school student. Is Jaynes’ PT:LOS able to be read by moi, given that I know basic set theory? If not, can anyone recommend a different textbook?
Jayne’s book probably requires a university undergraduate-level familiarity with probability theory to fully appreciate.
I’d say that for the time being you don’t need to worry about bayesianism vs. frequentism. Just learn the basics of probability theory and learn how to solve problems.
Thanks for being the one commenter who told me how tough the book is—I’m leaving it for now, and the below recommendation of ‘Understanding Uncertainty’ was very useful for understanding what a probability is. After that, I’ve got some basic probability textbooks waiting to go. Cheers.
It’s worth knowing that what Jaynes calls “probability” everyone else calls “statistics.”
Generally, “probability theory” means studying well-specified random models. In some sense this is frequentist, but in another sense the distinction does not apply. Whereas “statistics” is about subjective ignorance.
And simulation theory is kinda the opposite of statistics—whereas in statistics you deduce the distribution from sample data, in simulation you compute plausible sample data from a given distribution.
If you’re looking for an elementary introduction to Bayesian probability theory, I recommend Dennis Lindley’s Understanding Uncertainty. A lot more accessible than Jaynes, but not dumbed down. It’s informal, but it covers a number of quite sophisticated topics.
Lindley is one of the architects of the Bayesian conspiracy.
Given that PT:LOS is free online you can just try reading it. Even if you don’t understand all the maths (do you know some calculus?) you’ll still be able to read his verbal explanations of things, which will give you a good idea of the distinction between frequentist statistics and Bayes.
IIRC the version that’s online is not the same as the dead-tree version you can buy; the latter has extra material and bugfixes. (I do, none the less, think reading the online version is a good way for Benito to determine whether he finds it approachable.)
With math, it’s useful to be able to distinguish books you can’t understand because you’re missing prerequisite knowledge from books you can’t understand because you just aren’t reading them carefully enough. The prevailing wisdom seems to be that you can’t really expect to be able to follow Jaynes through if you pick it up as your first serious textbook on probability.
Agh. Please do not abuse English (or French) this way; what did they ever do to you? What you want to say is “Can I understand Jaynes’s PT:LOS?” This places the action where it belongs, with a human. A book is not “able to be read” by anyone. I am able to type, because I can perform the action of moving my fingers on the keyboard. Being read is not an action; consequently there is no such thing as “able to be read”. And even if there were, a book would not have that ability, because books do not perform actions. Additionally, that is one of the ugliest passive-voice constructs I’ve ever seen; and I’ve read quite a bit of unpublished academic writing. (And if you think the average journal article is awful, you should see what they’re like before the internal reviewers exercise their judgement, such as it is.) Finally, ‘moi’ for ‘me’ might have been archly funny or ironically pretentious the first two or three times it was used, in the sixties. The eighteen sixties.
Thumbs up to Benito for having the interest in these topics at that age. Rolf, why the rant against him? We should be encouraging to young people interested in rationality and bayesian probability.
Rolf’s comment is a fine example of the aphorism ‘praise should be delivered in public, criticism in private’. When I spot someone making a grammar error or formatting error or other minor error, I try to PM them rather than make a public comment. For two reasons:
People really don’t care, and a minor correction shouldn’t permanently clutter up comment threads. People reading Benito’s request for help don’t care whether people dislike the French. Yes, Rolf is right that it’s a little annoying and offputting. But if people don’t want to read his gratuitous use of French, they especially don’t want to read 5 or 15 comments debating it. So criticizing him with a public comment is wasteful of other peoples’ time.
Criticizing like that in public is especially likely to make someone slightly angry or to lash back or ignore it. So criticizing him with a public comment is less likely to accomplish the claimed goal of improving his writing.
I’ll note that, whilst I found Rolf’s comment mildly musing, it did not have a significant effect on the probability of me speaking like that in the future.
Gwern mentioning more in passing that it was a little annoying and off-putting, without being aggressive or rude about it, has affected me—I wasn’t aware it was either. I probably won’t use it again.
I’ve been teaching myself the basics of probability theory (I’m sixteen) but I’m having trouble on the first step. My basic definitions of probabilities are all frequentist, and I don’t know a good Bayesian source appropriate for a secondary school student. Is Jaynes’ PT:LOS able to be read by moi, given that I know basic set theory? If not, can anyone recommend a different textbook?
Jayne’s book probably requires a university undergraduate-level familiarity with probability theory to fully appreciate.
I’d say that for the time being you don’t need to worry about bayesianism vs. frequentism. Just learn the basics of probability theory and learn how to solve problems.
Thanks for being the one commenter who told me how tough the book is—I’m leaving it for now, and the below recommendation of ‘Understanding Uncertainty’ was very useful for understanding what a probability is. After that, I’ve got some basic probability textbooks waiting to go. Cheers.
It’s worth knowing that what Jaynes calls “probability” everyone else calls “statistics.”
Generally, “probability theory” means studying well-specified random models. In some sense this is frequentist, but in another sense the distinction does not apply. Whereas “statistics” is about subjective ignorance.
That terminology sounds strange to me.
I define statistics as a toolbox of methods to deal with uncertainty.
And simulation theory is kinda the opposite of statistics—whereas in statistics you deduce the distribution from sample data, in simulation you compute plausible sample data from a given distribution.
If you’re looking for an elementary introduction to Bayesian probability theory, I recommend Dennis Lindley’s Understanding Uncertainty. A lot more accessible than Jaynes, but not dumbed down. It’s informal, but it covers a number of quite sophisticated topics.
Lindley is one of the architects of the Bayesian conspiracy.
This recommendation has helped me out a lot, I might do a write-up of the book as a LW post at some point in the future. Thanks.
Given that PT:LOS is free online you can just try reading it. Even if you don’t understand all the maths (do you know some calculus?) you’ll still be able to read his verbal explanations of things, which will give you a good idea of the distinction between frequentist statistics and Bayes.
IIRC the version that’s online is not the same as the dead-tree version you can buy; the latter has extra material and bugfixes. (I do, none the less, think reading the online version is a good way for Benito to determine whether he finds it approachable.)
Indeed. (Although the dead-tree version doesn’t have that much extra material. It mostly just has the “—Much more here!!!--”″ notices deleted.)
A good way to find out would be to try reading it.
With math, it’s useful to be able to distinguish books you can’t understand because you’re missing prerequisite knowledge from books you can’t understand because you just aren’t reading them carefully enough. The prevailing wisdom seems to be that you can’t really expect to be able to follow Jaynes through if you pick it up as your first serious textbook on probability.
Agh. Please do not abuse English (or French) this way; what did they ever do to you? What you want to say is “Can I understand Jaynes’s PT:LOS?” This places the action where it belongs, with a human. A book is not “able to be read” by anyone. I am able to type, because I can perform the action of moving my fingers on the keyboard. Being read is not an action; consequently there is no such thing as “able to be read”. And even if there were, a book would not have that ability, because books do not perform actions. Additionally, that is one of the ugliest passive-voice constructs I’ve ever seen; and I’ve read quite a bit of unpublished academic writing. (And if you think the average journal article is awful, you should see what they’re like before the internal reviewers exercise their judgement, such as it is.) Finally, ‘moi’ for ‘me’ might have been archly funny or ironically pretentious the first two or three times it was used, in the sixties. The eighteen sixties.
Relevant SMBC.
Thumbs up to Benito for having the interest in these topics at that age. Rolf, why the rant against him? We should be encouraging to young people interested in rationality and bayesian probability.
Rolf’s comment is a fine example of the aphorism ‘praise should be delivered in public, criticism in private’. When I spot someone making a grammar error or formatting error or other minor error, I try to PM them rather than make a public comment. For two reasons:
People really don’t care, and a minor correction shouldn’t permanently clutter up comment threads. People reading Benito’s request for help don’t care whether people dislike the French. Yes, Rolf is right that it’s a little annoying and offputting. But if people don’t want to read his gratuitous use of French, they especially don’t want to read 5 or 15 comments debating it. So criticizing him with a public comment is wasteful of other peoples’ time.
Criticizing like that in public is especially likely to make someone slightly angry or to lash back or ignore it. So criticizing him with a public comment is less likely to accomplish the claimed goal of improving his writing.
I’ll note that, whilst I found Rolf’s comment mildly musing, it did not have a significant effect on the probability of me speaking like that in the future.
Gwern mentioning more in passing that it was a little annoying and off-putting, without being aggressive or rude about it, has affected me—I wasn’t aware it was either. I probably won’t use it again.