Pragmatism is worth studying from a pragmatic perspective. Materialism is worth studying from a materialist perspective. But postmodernism is not worth studying from a postmodernist perspective, so be careful to choose the right philosophies to begin with. Bwa ha ha.
Seriously though, the Less Wrong sequences are great. Eliezer started with math (Bayesianism) and science (cognitive science) and went ahead and pwned a huge swath of philosophy by staying within the well-established mathematical and cognitive frameworks. Philosophical subjects like ‘aesthetics’ are still difficult. (Schmidhuber’s work on it is interesting though, and he approaches it from a super-simplified computer science perspective.) But old dilemmas like ‘free will’ and ‘moral realism vs. antirealism’ are dissolved.
Er, I think so. The kind of person who reads Less Wrong is probably the kind of person who takes care to search Wikipedia or the SEP for words like ‘Bayesianism’ and get a feel for the relevant literature.
One of my favourite aspects of LW is knowing that, even if I forget to linkify an unusual or obscure name or reference, I will not get some Mr. McLazy replying with “What is X? Never heard of it”.
Frequentists accept all of the math behind Bayes Theorem, but are not Bayesians. The interpretation of probability is a philosophical subject, not something that is a straightforward consequence of the math.
Bayesianism is applied Bayesian probability theory.
Bayesianism is applied probability theory with additional epistemological premises. You can accept the former while rejecting the latter.
Note: I am out of my depth, and simply repeating (probably incorrectly) cached thoughts said by people smarter than me.
The interpretation of probability is a philosophical subject, not something that is a straightforward consequence of the math.
I was under the impression that there are sound decision theoretic and axiomatic justifications for the notion of subjective probability. Also, the sequences themselves provide pretty good justification. And if you want to quibble over the axioms of Cox’s theorems, that seems to me squarely within the domain of mathematics.
Philosophy is the name given to the search for well-defined questions. The frequentist/Bayesian dilemma seems to me well-defined enough to be considered within the domain of mathematics. Epistemology just barely intersects with it.
I was under the impression that there are sound decision theoretic and axiomatic justifications for the notion of subjective probability. Also, the sequences themselves provide pretty good justification.
I agree that there are good arguments for Bayesianism. I disagree that most of the premises of those arguments are mathematical premises rather than epistemological (philosophical) premises.
I agree that there are epistemological problems in the foundations, but they seem to me mild enough to refute frequentism. I’m not really sure what frequentism is, though (other than the position that one should not speak of the probability of a hypothesis). Can you spell out what you think the coherent frequentist position is? I won’t hold it against you or frequentism if you say no.
(If one just speaks of beliefs, maybe there is a coherent frequentist position that evades Cox’s theorem, but frequentists hold that we make decisions [ETA: a metaphysical but not epistemological assumption]; and this should be enough to force probabilistic beliefs.)
I’m not really sure what frequentism is, though...Can you spell out what you think the coherent frequentist position is?
I’m not sure either, as I’ve confessed before, but here’s how it seems to me: whereas Bayesians view beliefs as “thermometer readings” that go up and down as new information comes in, frequentists view beliefs as “certificates” that have to be “earned” through specific rituals, and are subject to periodic renewal.
Since (in view of results such as Cox’s theorem) one can’t really deny that probability theory is the appropriate mathematical model for belief if there is any appropriate model at all, it seems that what frequentists actually object to, deep down, is the idea of personal belief as the object of formalization. They are only comfortable with (“official”) belief-formation as a social process. In their view (so I propose), you are only allowed to make probabilistic claims after you perform a well-defined experiment of some kind (specifically, of the kind in which relative frequencies of the events you’re interested in can be observed).
Frequentists don’t have a defined view of beliefs, they have a defined view of probability, namely that probability statements about events are statements about the limit of that event’s frequency in an arbitrary number of trials. The Bayesian position is that probability can be understood as degree of belief. The Bayesian epistemological position is that all beliefs ought to be understood as probability statements. There is no Frequentist epistemology, a frequentist could hold any epistemology they wanted more or less.
I understand that this is the standard story, but don’t find it satisfactory; hence was presenting my own “deeper” explanation (which I still regard as tentative).
After all, based only on what you’ve said, a frequentist could also be a Bayesian! This doesn’t seem right, or at least doesn’t seem to account for the fact that there seems to be a controversy between “frequentists” on the one hand, and “Bayesians” on the other.
I don’t see how a frequentist could Bayesian, to hold a Bayesian epistemology one would also have to adhere to a Bayesian theory of probability. Since frequentists don’t hold such a theory that can not hold Bayesian epistemology.
As per my original comment, someone could agree that probabilities represent beliefs but also hold that the content of (“legitimate”) beliefs consists only of statements about frequencies. This would allow them to simultaneously hold a frequentist interpretation of probability and a Bayesian epistemology. (As you said, “There is no Frequentist epistemology, a frequentist could hold any epistemology they wanted more or less.”)
I’m not a Frequentist (so I won’t bother writing up a justification for the position), but non-Bayesians like R. A. Fisher, Jerzy Neyman, and Egon Pearson didn’t just dogmatically refuse to accept the conclusion of a valid mathematical argument. They denied the truth of Bayesianism’s epistemological premises (note: I disagree with their judgment in this case). Not one of them denied that Baye’s Rule could be derived from the very definition of conditional probability (which is a straightforward consequence of the mathematics).
My comment was intended to point out that it takes more than standard probability theory and deduction to get you to Bayesianism. Additional premises (from outside of mathematics) must be present (at least implicitly).
Not one of them denied that Baye’s Rule could be derived from the very definition of conditional probability (which is a straightforward consequence of the mathematics).
That’s a reasonable response to Will’s first two comments, but [ETA: not] as a response to his third comment, mentioning Cox’s theorem, or my comment, mentioning decision theory. I don’t blame you for not knowing whether they had a coherent system of beliefs, but I do blame you for this non sequitur.
ETA: maybe that would be reasonable if you just substituted Cox for Bayes, but only if these frequentists explicitly rejected their contemporary Cox, rather than just ignored him.
Pragmatism is worth studying from a pragmatic perspective. Materialism is worth studying from a materialist perspective. But postmodernism is not worth studying from a postmodernist perspective, so be careful to choose the right philosophies to begin with. Bwa ha ha.
Seriously though, the Less Wrong sequences are great. Eliezer started with math (Bayesianism) and science (cognitive science) and went ahead and pwned a huge swath of philosophy by staying within the well-established mathematical and cognitive frameworks. Philosophical subjects like ‘aesthetics’ are still difficult. (Schmidhuber’s work on it is interesting though, and he approaches it from a super-simplified computer science perspective.) But old dilemmas like ‘free will’ and ‘moral realism vs. antirealism’ are dissolved.
People realize Bayesian epistemology was a philosophy long before Eliezer, right? It’s a fairly large area of study in philosophy of science.
Er, I think so. The kind of person who reads Less Wrong is probably the kind of person who takes care to search Wikipedia or the SEP for words like ‘Bayesianism’ and get a feel for the relevant literature.
One of my favourite aspects of LW is knowing that, even if I forget to linkify an unusual or obscure name or reference, I will not get some Mr. McLazy replying with “What is X? Never heard of it”.
Bayesianism is an epistemological position. Epistemology is a branch of philosophy. So, Eliezer did not pwn philosophy with “math”.
Bayesianism is applied Bayesian probability theory. Bayesian probability theory is math. Which of those two propositions do you disagree with?
Frequentists accept all of the math behind Bayes Theorem, but are not Bayesians. The interpretation of probability is a philosophical subject, not something that is a straightforward consequence of the math.
Bayesianism is applied probability theory with additional epistemological premises. You can accept the former while rejecting the latter.
Note: I am out of my depth, and simply repeating (probably incorrectly) cached thoughts said by people smarter than me.
I was under the impression that there are sound decision theoretic and axiomatic justifications for the notion of subjective probability. Also, the sequences themselves provide pretty good justification. And if you want to quibble over the axioms of Cox’s theorems, that seems to me squarely within the domain of mathematics.
Philosophy is the name given to the search for well-defined questions. The frequentist/Bayesian dilemma seems to me well-defined enough to be considered within the domain of mathematics. Epistemology just barely intersects with it.
I agree that there are good arguments for Bayesianism. I disagree that most of the premises of those arguments are mathematical premises rather than epistemological (philosophical) premises.
I agree that there are epistemological problems in the foundations, but they seem to me mild enough to refute frequentism. I’m not really sure what frequentism is, though (other than the position that one should not speak of the probability of a hypothesis). Can you spell out what you think the coherent frequentist position is? I won’t hold it against you or frequentism if you say no.
(If one just speaks of beliefs, maybe there is a coherent frequentist position that evades Cox’s theorem, but frequentists hold that we make decisions [ETA: a metaphysical but not epistemological assumption]; and this should be enough to force probabilistic beliefs.)
I’m not sure either, as I’ve confessed before, but here’s how it seems to me: whereas Bayesians view beliefs as “thermometer readings” that go up and down as new information comes in, frequentists view beliefs as “certificates” that have to be “earned” through specific rituals, and are subject to periodic renewal.
Since (in view of results such as Cox’s theorem) one can’t really deny that probability theory is the appropriate mathematical model for belief if there is any appropriate model at all, it seems that what frequentists actually object to, deep down, is the idea of personal belief as the object of formalization. They are only comfortable with (“official”) belief-formation as a social process. In their view (so I propose), you are only allowed to make probabilistic claims after you perform a well-defined experiment of some kind (specifically, of the kind in which relative frequencies of the events you’re interested in can be observed).
See Science Doesn’t Trust Your Rationality.
Frequentists don’t have a defined view of beliefs, they have a defined view of probability, namely that probability statements about events are statements about the limit of that event’s frequency in an arbitrary number of trials. The Bayesian position is that probability can be understood as degree of belief. The Bayesian epistemological position is that all beliefs ought to be understood as probability statements. There is no Frequentist epistemology, a frequentist could hold any epistemology they wanted more or less.
I understand that this is the standard story, but don’t find it satisfactory; hence was presenting my own “deeper” explanation (which I still regard as tentative).
After all, based only on what you’ve said, a frequentist could also be a Bayesian! This doesn’t seem right, or at least doesn’t seem to account for the fact that there seems to be a controversy between “frequentists” on the one hand, and “Bayesians” on the other.
I don’t see how a frequentist could Bayesian, to hold a Bayesian epistemology one would also have to adhere to a Bayesian theory of probability. Since frequentists don’t hold such a theory that can not hold Bayesian epistemology.
As per my original comment, someone could agree that probabilities represent beliefs but also hold that the content of (“legitimate”) beliefs consists only of statements about frequencies. This would allow them to simultaneously hold a frequentist interpretation of probability and a Bayesian epistemology. (As you said, “There is no Frequentist epistemology, a frequentist could hold any epistemology they wanted more or less.”)
I’m not a Frequentist (so I won’t bother writing up a justification for the position), but non-Bayesians like R. A. Fisher, Jerzy Neyman, and Egon Pearson didn’t just dogmatically refuse to accept the conclusion of a valid mathematical argument. They denied the truth of Bayesianism’s epistemological premises (note: I disagree with their judgment in this case). Not one of them denied that Baye’s Rule could be derived from the very definition of conditional probability (which is a straightforward consequence of the mathematics).
My comment was intended to point out that it takes more than standard probability theory and deduction to get you to Bayesianism. Additional premises (from outside of mathematics) must be present (at least implicitly).
That’s a reasonable response to Will’s first two comments, but [ETA: not] as a response to his third comment, mentioning Cox’s theorem, or my comment, mentioning decision theory. I don’t blame you for not knowing whether they had a coherent system of beliefs, but I do blame you for this non sequitur.
ETA: maybe that would be reasonable if you just substituted Cox for Bayes, but only if these frequentists explicitly rejected their contemporary Cox, rather than just ignored him.