“We aren’t enchanted by Bayesian methods merely because they’re beautiful. The beauty is a side effect. Bayesian theorems are elegant, coherent, optimal, and provably unique because they are laws.”
This seems deeply mistaken. Why should we believe that bayesian formulations are any more inherently “lawlike” than frequentist formulations? Both derive their theorems from within strict formal systems which begin with unchanging first principles. The fundamental difference between Bayesians and Frequentists seems to stem from different ontological assumptions about the nature of a probability distribution (Frequentists imagine a distribution as a set of possible outcomes which would have occurred under different realizations of our world, whereas Bayesians imagine a distribution as a description of single subjective mental state regarding a single objective world about which we are uncertain).
Moreover, doesn’t Cox’s Theorem imply that at a sufficient level of abstraction, any Bayesian derivation could (at least in principle) be creatively re-framed as a Frequentist derivation, since both must map (at some level) onto the basic rules of probability? It seems to me, that as far as the pure “math” is concerned, both frameworks have equal claim to “lawlike” status.
It therefore seems that what drives Eliezer (and many others, myself included) towards Bayesian formulations is a type of (dare I say it?) bias towards a certain kind of beauty which he has cleverly re-labeled as “law.”
If Bayesian derivation is a frequentist derivation, it does not mean that any frequentist derivation is necessarily equivalent to Bayesian. Mr. Yudkowsky claims, more or less, that Bayesian derivation is equivalent to the ideal frequentist derivation.
Elizer says:
“We aren’t enchanted by Bayesian methods merely because they’re beautiful. The beauty is a side effect. Bayesian theorems are elegant, coherent, optimal, and provably unique because they are laws.”
This seems deeply mistaken. Why should we believe that bayesian formulations are any more inherently “lawlike” than frequentist formulations? Both derive their theorems from within strict formal systems which begin with unchanging first principles. The fundamental difference between Bayesians and Frequentists seems to stem from different ontological assumptions about the nature of a probability distribution (Frequentists imagine a distribution as a set of possible outcomes which would have occurred under different realizations of our world, whereas Bayesians imagine a distribution as a description of single subjective mental state regarding a single objective world about which we are uncertain).
Moreover, doesn’t Cox’s Theorem imply that at a sufficient level of abstraction, any Bayesian derivation could (at least in principle) be creatively re-framed as a Frequentist derivation, since both must map (at some level) onto the basic rules of probability? It seems to me, that as far as the pure “math” is concerned, both frameworks have equal claim to “lawlike” status.
It therefore seems that what drives Eliezer (and many others, myself included) towards Bayesian formulations is a type of (dare I say it?) bias towards a certain kind of beauty which he has cleverly re-labeled as “law.”
If Bayesian derivation is a frequentist derivation, it does not mean that any frequentist derivation is necessarily equivalent to Bayesian. Mr. Yudkowsky claims, more or less, that Bayesian derivation is equivalent to the ideal frequentist derivation.