I haven’t read this in detail but one very quick comment: Cox’s Theorem is a representation theorem showing that coherent belief states yield classical probabilities, it’s not the same as the dutch-book theorem at all. E.g. if you want to represent probabilities using log odds, they can certain relate to each other coherently (since they’re just transforms of classical probabilities), but Cox’s Theorem will give you the classical probabilities right back out again. Jaynes cites a special case of Cox in PT:TLOS which is constructive at the price of assuming probabilities are twice differentiable, and I actually tried it with log odds and got the classical probabilities right back out—I remember being pretty impressed with that, and had this enlightenment experience wherein I went to seeing probability theory as a kind of relational structure in uncertainty.
I also quickly note that the worst-case scenario often amounts to making unfair assumptions about “randomization” wherein adversaries can always read the code of deterministic agents but non-deterministic agents have access to hidden sources of random numbers. E.g. http://lesswrong.com/lw/vq/the_weighted_majority_algorithm/
Good catch on Cox’s theorem; that is now fixed. Do you know if the dutch book argument corresponds to a named theorem?
I’m not sure exactly how your comment about deterministic vs. non-deterministic agents is meant to apply to the arguments I’ve advanced here (although I suppose you will clarify after you’re done reading).
Separately, I disagree that the assumptions are unfair; I think of it as a particularly crisp abstraction of the actual situation you care about. As long as pseudo-random generators exist and you can hide your source of randomness, you can guarantee that no adversary can predict your random bits; if you could usefully make the same guarantee about other aspects of your actions without recourse to a PRG then I would happily incorporate that into the set of assumptions, but in practice it is easiest to just work in terms of a private source of randomness. Besides, I think that the use of this formalism has been amply validated by its intellectual fruits (see the cited network flow application as one example, or the Arora, Hazan, and Kale reference).
Good catch on Cox’s theorem; that is now fixed. Do you know if the dutch book argument corresponds to a named theorem?
There is a whole class of dutch book arguments, so I’m not sure which one you mean by the dutch book argument.
In any case, Susan Vineberg’s formulation of the Dutch Book Theorem goes like this:
Given a set of betting quotients that fails to satisfy the probability axioms, there is a set of bets with those quotients that guarantees a net loss to one side.
Then you might think you could have inconsistent betting prices that would harm the person you bet with, but not you, which sounds fine.
Rather: “If your betting prices don’t obey the laws of probability theory, then you will either accept combinations of bets that are sure losses, or pass up combinations of bets that are sure gains.”
Well, Cox’s Theorem still assumes that you’re representing belief-strengths with real numbers in the first place. Really you should go back to Savage’s Theorem… :)
I’ve tried to do something similar with odds once, but the assumption about (AB|C) = F[(A|C), (B|AC)] made me give up.
Indeed, one can calculate O(AB|C) given O(A|C) and O(B|AC) but the formula isn’t pretty. I’ve tried to derive that function but failed. It was not until I appealed to the fact that O(A)=P(A)/(1-P(A)) that I managed to infer this unnatural equation about O(AB|C), O(A|C) and O(B|AC).
And this use of classical probabilities, of course, completely defeats the point of getting classical probabilities from the odds via Cox’s Theorem!
Did I miss something?
By the way, are there some other interesting natural rules of inference besides odds and log odds which are isomorphic to the rules of probability theory? (Judea Pearl mentioned something about MYCIN certainty factor, but I was unable to find any details)
EDIT: You can view the CF combination rules here, but I find it very difficult to digest. Also, what about initial assignment of certainty?
I haven’t read this in detail but one very quick comment: Cox’s Theorem is a representation theorem showing that coherent belief states yield classical probabilities, it’s not the same as the dutch-book theorem at all. E.g. if you want to represent probabilities using log odds, they can certain relate to each other coherently (since they’re just transforms of classical probabilities), but Cox’s Theorem will give you the classical probabilities right back out again. Jaynes cites a special case of Cox in PT:TLOS which is constructive at the price of assuming probabilities are twice differentiable, and I actually tried it with log odds and got the classical probabilities right back out—I remember being pretty impressed with that, and had this enlightenment experience wherein I went to seeing probability theory as a kind of relational structure in uncertainty.
I also quickly note that the worst-case scenario often amounts to making unfair assumptions about “randomization” wherein adversaries can always read the code of deterministic agents but non-deterministic agents have access to hidden sources of random numbers. E.g. http://lesswrong.com/lw/vq/the_weighted_majority_algorithm/
Good catch on Cox’s theorem; that is now fixed. Do you know if the dutch book argument corresponds to a named theorem?
I’m not sure exactly how your comment about deterministic vs. non-deterministic agents is meant to apply to the arguments I’ve advanced here (although I suppose you will clarify after you’re done reading).
Separately, I disagree that the assumptions are unfair; I think of it as a particularly crisp abstraction of the actual situation you care about. As long as pseudo-random generators exist and you can hide your source of randomness, you can guarantee that no adversary can predict your random bits; if you could usefully make the same guarantee about other aspects of your actions without recourse to a PRG then I would happily incorporate that into the set of assumptions, but in practice it is easiest to just work in terms of a private source of randomness. Besides, I think that the use of this formalism has been amply validated by its intellectual fruits (see the cited network flow application as one example, or the Arora, Hazan, and Kale reference).
There is a whole class of dutch book arguments, so I’m not sure which one you mean by the dutch book argument.
In any case, Susan Vineberg’s formulation of the Dutch Book Theorem goes like this:
Yes, that is the one I had in mind. Thanks!
Then you might think you could have inconsistent betting prices that would harm the person you bet with, but not you, which sounds fine.
Rather: “If your betting prices don’t obey the laws of probability theory, then you will either accept combinations of bets that are sure losses, or pass up combinations of bets that are sure gains.”
Well, Cox’s Theorem still assumes that you’re representing belief-strengths with real numbers in the first place. Really you should go back to Savage’s Theorem… :)
I’ve tried to do something similar with odds once, but the assumption about (AB|C) = F[(A|C), (B|AC)] made me give up.
Indeed, one can calculate O(AB|C) given O(A|C) and O(B|AC) but the formula isn’t pretty. I’ve tried to derive that function but failed. It was not until I appealed to the fact that O(A)=P(A)/(1-P(A)) that I managed to infer this unnatural equation about O(AB|C), O(A|C) and O(B|AC).
And this use of classical probabilities, of course, completely defeats the point of getting classical probabilities from the odds via Cox’s Theorem!
Did I miss something?
By the way, are there some other interesting natural rules of inference besides odds and log odds which are isomorphic to the rules of probability theory? (Judea Pearl mentioned something about MYCIN certainty factor, but I was unable to find any details)
EDIT: You can view the CF combination rules here, but I find it very difficult to digest. Also, what about initial assignment of certainty?
EDIT2: Nevermind, I found an adequate summary ( http://www.idi.ntnu.no/~ksys/NOTES/CF-model.html ) of the model and pdf ( http://uai.sis.pitt.edu/papers/85/p9-heckerman.pdf ) about probabilistic interpretations of CF. It seems to be an interesting example of not-obviously-Bayesian system of inference, but it’s not exactly an example you would give to illustrate the point of Cox’s theorem.