I think most LW people take a Bayesian view of probability, where probability is a consistent, numerical measure of how confident we are that a proposition is true, updated according to Bayes rule, etc. You’re advocating the mathematical view of probability, where probability really means “probability measure”, ie the measure on a measure space of measure 1.
It’s not that one of these views is right and the other is wrong. They’re actually describing two slightly differentthings. Measure-theoretic probability is something we can have a completely rigorous mathematical theory of, because it’s assumptions are technical and precise. But having a mathematical theory doesn’t necessarily mean you can apply it to the real world. You need knowledge and judgment to determine what real-world phenomena are modeled by your theory and how well. Measure theoretic probability models card games very well (measure space = the uniform distribution over the 52! ways that the deck could be shuffled) if the dealer is honest, but it doesn’t help you decide weather to trust the dealer.
Bayesian probability is a less rigorous, but more directly-applicable-to-real-life theory. It shares a lot of terminology and theorems with measure theoretic probability, but it isn’t quite the same thing. In particular, in Bayesian probability you don’t have a probability space, so when you see people here talking probability without specifying the space, it’s not an error, they’re just being Bayesian.
Jayne’s “Probability Theory” is a great book on Bayesian probability theory, but if you’ve got a Mathematical education, which I suspect you might, it’s going to piss you off. Just ignore the parts where Jaynes bloviates about things he doesn’t understand, and learn from the parts where he teaches the things he does. Most of the book is the latter.
Er… So, where does the measure-theoretic definition of probability become incompatible with “Bayesian probability” you talk about? Can you give a reference that supports your position? (I understand there are disputes on foundations, but representationally these all seem to be exactly the same thing.)
Who said they were incompatible? I only said they’re different. In measure-theoretic probability theory, you start with a space, in Bayseian, you don’t. In measure-theory land the propositions that get assigned probabilities must be subsets of the space, in Bayes-land they can be anything that’s true or false (or will be in the future) in the real world.
The difference between the two is not a dispute on foundations. They really are two different, but overlapping theories. Measure-theoretic probability theory is a formal mathematical theory, like group theory or point-set topology, It’s a set of theorems about mathematical objects (probability measures) that satisfy certain axioms. Those objects may be good models of something in real life, or they may not. Either way the theorems are still true. For a reference on this topic, see, well, any book on measure theory. There are lots of them. Here’s one
Bayesian theory is just not that formal. What are the axioms of Bayesian theory? What propositions are allowed? How do you select Priors? Bayesian probability may use a lot of math, but math isn’t what it is. It’s more like physics than group theory.
Bayesian probability may use a lot of math, but math isn’t what it is.
Yet it seems that math is what it should be. Bayesian probability, as it’s used in probabilistic inference, is usually founded on the same Kolmogorov axioms, standard mathematical probability theory. I don’t see any problems with the mathematical part, I dispute your characterization of Bayesian probability as an inherently informal theory (hence it was taken in quotation marks in my comment).
I think smoofra is talking about the same sorts of things Jaynes is when he writes:
The danger here is particularly great because mathematicians generally regard these limit theorems as the most important and sophisticated fruits of probability theory, and have a tendency to use language which implies that they are proving properties of the real world. Our point is that these theorems are valid properties of the abstract mathematical model that was defined and analyzed [emphasis in original]. The issue is: to what extent does that model resemble the real world? It is probably safe to say that no limit theorem is directly applicable in the real world, simply because no mathematical model captures every circumstance that is relevant in the real world.
Jaynes aggressively scorns abstract mathematics. I love abstract mathematics. We both agree that just because you have a model or a theorem, it doesn’t necessarily apply to the real world.
I come to quote Jaynes, not to praise him; the scorn that men write lives after them, the good is oft interred with their bones—let it not be thus with Jaynes.
Yep, you shouldn’t wirehead yourself into developing a theory about the mathematical formalism, you should instead develop a theory about the world. But the theory that you develop should be mathematical where possible.
Arguably there may be, if it can be shown that you normatively should worry only about the real world, even if what you are doing in the real world is thinking math.
if it can be shown that you normatively should worry only about the real world,
It can’t be. Not in any system of norms I would give a fig about. Art, Fiction, and Math are worthwhile. They don’t have to be useful. If you disagree with that, then we simply have different utility functions, and there’s no point in arguing further.
You are seeing “useful” too narrowly. I only stated that whatever you consider “useful”, it’s probably a statement exclusively about the real world, and “doing math” is one of the activities in the real world. I don’t see how you could place Art in the same cached thought, since it was remarked many times that you shouldn’t go Spock.
Any theory about the real world is inherently informal.
Do you disagree that Bayesian probability theory is about as informal as physics, or do you disagree with my characterization of physics as informal? If it’s the latter, then we don’t disagree on anything except the meaning of words.
A theory about the real world may be perfectly formal, it just won’t have a perfectly formal applicability proof. On the other hand, if you can show that a theory is applicable with probability of 1-2^{-10000}, it’s as good as formally proven to apply.
I disagree that it’s correct terminology to call a theory informal, just because it’s can’t be formally proven to apply to the real world.
It’s not the lack of a proof that makes it informal, it’s that the elements themselves of the theory aren’t precisely, formally, mathematically defined. A valid proposition in measure-theoretic probability is a subset of the measure space. nothingelse will do. Propositions in Bayseian probability are written in natural language, about events in the real world.
I’m using the word “formal” in the sense that it is used in mathematics. If you’re going to say that propositions written in natural language, about events in the real world are “formal” in that sense, then you’re just refusing to communicate.
I think most LW people take a Bayesian view of probability, where probability is a consistent, numerical measure of how confident we are that a proposition is true, updated according to Bayes rule, etc. You’re advocating the mathematical view of probability, where probability really means “probability measure”, ie the measure on a measure space of measure 1.
It’s not that one of these views is right and the other is wrong. They’re actually describing two slightly differentthings. Measure-theoretic probability is something we can have a completely rigorous mathematical theory of, because it’s assumptions are technical and precise. But having a mathematical theory doesn’t necessarily mean you can apply it to the real world. You need knowledge and judgment to determine what real-world phenomena are modeled by your theory and how well. Measure theoretic probability models card games very well (measure space = the uniform distribution over the 52! ways that the deck could be shuffled) if the dealer is honest, but it doesn’t help you decide weather to trust the dealer.
Bayesian probability is a less rigorous, but more directly-applicable-to-real-life theory. It shares a lot of terminology and theorems with measure theoretic probability, but it isn’t quite the same thing. In particular, in Bayesian probability you don’t have a probability space, so when you see people here talking probability without specifying the space, it’s not an error, they’re just being Bayesian.
Jayne’s “Probability Theory” is a great book on Bayesian probability theory, but if you’ve got a Mathematical education, which I suspect you might, it’s going to piss you off. Just ignore the parts where Jaynes bloviates about things he doesn’t understand, and learn from the parts where he teaches the things he does. Most of the book is the latter.
Er… So, where does the measure-theoretic definition of probability become incompatible with “Bayesian probability” you talk about? Can you give a reference that supports your position? (I understand there are disputes on foundations, but representationally these all seem to be exactly the same thing.)
Who said they were incompatible? I only said they’re different. In measure-theoretic probability theory, you start with a space, in Bayseian, you don’t. In measure-theory land the propositions that get assigned probabilities must be subsets of the space, in Bayes-land they can be anything that’s true or false (or will be in the future) in the real world.
The difference between the two is not a dispute on foundations. They really are two different, but overlapping theories. Measure-theoretic probability theory is a formal mathematical theory, like group theory or point-set topology, It’s a set of theorems about mathematical objects (probability measures) that satisfy certain axioms. Those objects may be good models of something in real life, or they may not. Either way the theorems are still true. For a reference on this topic, see, well, any book on measure theory. There are lots of them. Here’s one
http://www.amazon.com/Probability-Measure-3rd-Patrick-Billingsley/dp/0471007102/ref=sr_1_13?ie=UTF8&s=books&qid=1241625621&sr=1-13
Bayesian theory is just not that formal. What are the axioms of Bayesian theory? What propositions are allowed? How do you select Priors? Bayesian probability may use a lot of math, but math isn’t what it is. It’s more like physics than group theory.
Yet it seems that math is what it should be. Bayesian probability, as it’s used in probabilistic inference, is usually founded on the same Kolmogorov axioms, standard mathematical probability theory. I don’t see any problems with the mathematical part, I dispute your characterization of Bayesian probability as an inherently informal theory (hence it was taken in quotation marks in my comment).
I think smoofra is talking about the same sorts of things Jaynes is when he writes:
- PT:LOS, pp 65-66.
ADBOC
Jaynes aggressively scorns abstract mathematics. I love abstract mathematics. We both agree that just because you have a model or a theorem, it doesn’t necessarily apply to the real world.
edit: (ADBOC directed to jaynes, not to cyan)
I come to quote Jaynes, not to praise him; the scorn that men write lives after them, the good is oft interred with their bones—let it not be thus with Jaynes.
Yep, you shouldn’t wirehead yourself into developing a theory about the mathematical formalism, you should instead develop a theory about the world. But the theory that you develop should be mathematical where possible.
There’s nothing wrong with doing pure Math, if you know that’s what your doing.
Arguably there may be, if it can be shown that you normatively should worry only about the real world, even if what you are doing in the real world is thinking math.
It can’t be. Not in any system of norms I would give a fig about. Art, Fiction, and Math are worthwhile. They don’t have to be useful. If you disagree with that, then we simply have different utility functions, and there’s no point in arguing further.
You are seeing “useful” too narrowly. I only stated that whatever you consider “useful”, it’s probably a statement exclusively about the real world, and “doing math” is one of the activities in the real world. I don’t see how you could place Art in the same cached thought, since it was remarked many times that you shouldn’t go Spock.
Any theory about the real world is inherently informal.
Do you disagree that Bayesian probability theory is about as informal as physics, or do you disagree with my characterization of physics as informal? If it’s the latter, then we don’t disagree on anything except the meaning of words.
A theory about the real world may be perfectly formal, it just won’t have a perfectly formal applicability proof. On the other hand, if you can show that a theory is applicable with probability of 1-2^{-10000}, it’s as good as formally proven to apply.
I disagree that it’s correct terminology to call a theory informal, just because it’s can’t be formally proven to apply to the real world.
It’s not the lack of a proof that makes it informal, it’s that the elements themselves of the theory aren’t precisely, formally, mathematically defined. A valid proposition in measure-theoretic probability is a subset of the measure space. nothing else will do. Propositions in Bayseian probability are written in natural language, about events in the real world.
I’m using the word “formal” in the sense that it is used in mathematics. If you’re going to say that propositions written in natural language, about events in the real world are “formal” in that sense, then you’re just refusing to communicate.