As others have pointed out, there is no uniform probability distribution on a countable set. There are various generalisations of probability that drop or weaken the axiom of countable additivity, which have their uses, but one statistician’s conclusion is that you lose too many useful properties. On the other hand, writing a blog post to describe something as a lost cause suggests that it still has adherents. Googling /”finite additivity” probability/ turns up various attempts to drop countable additivity.
Another way of avoiding the axiom is to reject all infinities. There are then no countable sets to be countably additive over. This throws out almost all of current mathematics, and has attracted few believers.
In some computations involving probabilities, the axiom that the measure over the whole space is 1 plays no role. A notable example is the calculation of posterior probabilities from priors and data by Bayes’ Theorem:
The total measure of the prior cancels out of the numerator and denominator. This allows the use of “improper” priors that can have an infinite total measure, such as the one that assigns measure 1 to every integer and infinite measure to the set of all integers.
There can be a uniform probability distribution over an uncountable set, because there is no requirement for a probability distribution to be uncountably additive. Every sample drawn from the uniform distribution over the unit interval has a probability 0 of being drawn. This is just one of those things that one comes to understand by getting used to it, like square roots of −1, 0.999...=1, non-euclidean geometry, and so on.
As I recall, Teddy Seidenfeld is a fan of finite additivity.
Do you know why?
The recent thread on optional stopping and Bayes led me to this paper, which I see Seidenfeld is one of the authors of, which argues that countable additivity has bad consequences. But these consequences are a result of improper handling of limits, as Jaynes sets forth in his chapter 15. Seidenfeld and his coauthors go to great lengths (also here) exploring the negative consequences of finite additivity for Bayesian reasoning. They see this as a problem for Bayesian reasoning rather than for finite additivity. But I have not seen their motivation.
If you’re going to do probability on infinite spaces at all, finite additivity just seems to me to be an obviously wrong concept.
ETA: Here’s another paper by Seidenfeld, whose title does rather suggest that it is going to argue against finite additivity, but whose closing words decline to resolve the matter.
As others have pointed out, there is no uniform probability distribution on a countable set. There are various generalisations of probability that drop or weaken the axiom of countable additivity, which have their uses, but one statistician’s conclusion is that you lose too many useful properties. On the other hand, writing a blog post to describe something as a lost cause suggests that it still has adherents. Googling /”finite additivity” probability/ turns up various attempts to drop countable additivity.
Another way of avoiding the axiom is to reject all infinities. There are then no countable sets to be countably additive over. This throws out almost all of current mathematics, and has attracted few believers.
In some computations involving probabilities, the axiom that the measure over the whole space is 1 plays no role. A notable example is the calculation of posterior probabilities from priors and data by Bayes’ Theorem:
Posterior(H|D) = P(D|H) Prior(H) / Sum_H’ ( P(D|H’) Prior(H’) )
(H, H’ = hypothesis, D = data.)
The total measure of the prior cancels out of the numerator and denominator. This allows the use of “improper” priors that can have an infinite total measure, such as the one that assigns measure 1 to every integer and infinite measure to the set of all integers.
There can be a uniform probability distribution over an uncountable set, because there is no requirement for a probability distribution to be uncountably additive. Every sample drawn from the uniform distribution over the unit interval has a probability 0 of being drawn. This is just one of those things that one comes to understand by getting used to it, like square roots of −1, 0.999...=1, non-euclidean geometry, and so on.
As I recall, Teddy Seidenfeld is a fan of finite additivity. He does decision theory work, also.
Do you know why?
The recent thread on optional stopping and Bayes led me to this paper, which I see Seidenfeld is one of the authors of, which argues that countable additivity has bad consequences. But these consequences are a result of improper handling of limits, as Jaynes sets forth in his chapter 15. Seidenfeld and his coauthors go to great lengths (also here) exploring the negative consequences of finite additivity for Bayesian reasoning. They see this as a problem for Bayesian reasoning rather than for finite additivity. But I have not seen their motivation.
If you’re going to do probability on infinite spaces at all, finite additivity just seems to me to be an obviously wrong concept.
ETA: Here’s another paper by Seidenfeld, whose title does rather suggest that it is going to argue against finite additivity, but whose closing words decline to resolve the matter.