(I can’t be bothered to figure out the correct constants to normalize these. Should be fairly straightforward to calculate from the standard infinite-geometric-series-sum formula.)
Both of these functions have the same range, −1<x<1, but there is no overlap between the two functions. Their support is disjoint.
They are both (un-normalized) probability density functions, as per the names pdfA and pdfB. My apologies if that was unclear.
To be somewhat clearer: I was referring to the probability distributions described by these two probability density functions. They have the same range, but disjoint support, and so anything you drew from B could not have been drawn from A.
In other words, a (pathological) counterexample to “The range of the distributions is the same, so anything you draw from B could have been drawn from A”.
In mathematics, the range of a function may refer to either of two closely related concepts: The codomain of the function; The image of the function.
I meant the image. At least that’s what you call it for a function; I don’t know the terminology for distributions. Honestly I wasn’t thinking much about the word “range”, and should have simply said:
Anything you draw from B could have been drawn from A. And yet...
Before anyone starts on about how this statement isn’t well defined because the probability that you select any particular value from a continuous distribution, I’ll point out that I’ve never seen anyone draw a real number uniformly at random between 0 and 1 from a hat. Even if you are actually selecting from a continuous distribution, the observations we can make about it are finite, so the relevant probabilities are all finite.
I was assuming you meant range as in the statistical term (for a distribution, roughly, the maximum x for which pdf(x)≠0, minus the minimum x for which pdf(x)≠0).
Annoyingly, this is closer to the domain than it is the range, in function terminology.
I meant the image.
Are you sure? The range is a description of the possible outputs of the pdf, which means… almost nothing. Trivial counterexample if you do mean image:
Uniform distribution A between 0 and 0.5 (that is, 2 for 0..0.5, and 0 otherwise).
Uniform distribution B between 1.0 and 1.5 (that is, 2 for 1.0..1.5, and 0 otherwise).
Both of these distributions have the same image {0, 2}. And yet they are disjoint.
Honestly I wasn’t thinking much about the word “range”, and should have simply said: > Anything you draw from B could have been drawn from A.
There are many probability distributions where this is not the case. (Like the two uniform distributions A and B I give in this post.)
*****
Oh. You said you don’t know the terminology for distributions. Is it possible you’re under a misunderstanding of what a distribution is? It’s an “input” of a possible result, and an “output” of how probable that result is[1]. The output is not a result. The input is.
Oh. You said you don’t know the terminology for distributions. Is it possible you’re under a misunderstanding of what a distribution is? It’s an “input” of a possible result, and an “output” of how probable that result is.
Yup, it was that. I thought “possible values of the distribution”, and my brain output “range, like in functions”. I shall endeavor not to use a technical term when I don’t mean it or need it, because wow was this a tangent.
If I may ask, why didn’t you use the following (simpler imo) example: pmf_A(0) = 1 pmf_A(1) = 0 pmf_B(0) = 0 pmf_B(1) = 1
With that approach one can argue that the two PMFs have different ranges[1], and get rabbit-holed into a discussion of e.g. “is a uniform distribution from 0 to 1 with a range of −10 to 10 the same or different than a uniform distribution from 0 to 1 with a range of 0 to 1”.
This approach is more complex, but sidesteps that.
You draw an element at random from distribution A.
Or you draw an element at random from distribution B.
The range of the distributions is the same, so anything you draw from B could have been drawn from A. And yet...
This does not hold in pathological cases, I don’t think. As one example:
pdfA(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩...0,−1+1/24≤x<−1+1/231,−1+1/23≤x<−1+1/220,−1+1/22≤x<−1+1/211,−1+1/21≤x<1−1/210,1−1/21≤x<1−1/221,1−1/22≤x<1−1/230,1−1/23≤x<1−1/24...
pdfB(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩...1,−1+1/24≤x<−1+1/230,−1+1/23≤x<−1+1/221,−1+1/22≤x<−1+1/210,−1+1/21≤x<1−1/211,1−1/21≤x<1−1/220,1−1/22≤x<1−1/231,1−1/23≤x<1−1/24...
(I can’t be bothered to figure out the correct constants to normalize these. Should be fairly straightforward to calculate from the standard infinite-geometric-series-sum formula.)
Both of these functions have the same range, −1<x<1, but there is no overlap between the two functions. Their support is disjoint.
That’s a function, he was referring to a distribution
They are both (un-normalized) probability density functions, as per the names pdfA and pdfB. My apologies if that was unclear.
To be somewhat clearer: I was referring to the probability distributions described by these two probability density functions. They have the same range, but disjoint support, and so anything you drew from B could not have been drawn from A.
In other words, a (pathological) counterexample to “The range of the distributions is the same, so anything you draw from B could have been drawn from A”.
Wikipedia says:
I meant the image. At least that’s what you call it for a function; I don’t know the terminology for distributions. Honestly I wasn’t thinking much about the word “range”, and should have simply said:
Before anyone starts on about how this statement isn’t well defined because the probability that you select any particular value from a continuous distribution, I’ll point out that I’ve never seen anyone draw a real number uniformly at random between 0 and 1 from a hat. Even if you are actually selecting from a continuous distribution, the observations we can make about it are finite, so the relevant probabilities are all finite.
I was assuming you meant range as in the statistical term (for a distribution, roughly, the maximum x for which pdf(x)≠0, minus the minimum x for which pdf(x)≠0).
Annoyingly, this is closer to the domain than it is the range, in function terminology.
Are you sure? The range is a description of the possible outputs of the pdf, which means… almost nothing. Trivial counterexample if you do mean image:
Uniform distribution A between 0 and 0.5 (that is, 2 for 0..0.5, and 0 otherwise).
Uniform distribution B between 1.0 and 1.5 (that is, 2 for 1.0..1.5, and 0 otherwise).
Both of these distributions have the same image {0, 2}. And yet they are disjoint.
There are many probability distributions where this is not the case. (Like the two uniform distributions A and B I give in this post.)
*****
Oh. You said you don’t know the terminology for distributions. Is it possible you’re under a misunderstanding of what a distribution is? It’s an “input” of a possible result, and an “output” of how probable that result is[1]. The output is not a result. The input is.
...to way oversimplify, especially for continuous distributions.
Yup, it was that. I thought “possible values of the distribution”, and my brain output “range, like in functions”. I shall endeavor not to use a technical term when I don’t mean it or need it, because wow was this a tangent.
If I may ask, why didn’t you use the following (simpler imo) example: pmf_A(0) = 1 pmf_A(1) = 0 pmf_B(0) = 0 pmf_B(1) = 1
Or even the “Bit sequences” part of the post?
With that approach one can argue that the two PMFs have different ranges[1], and get rabbit-holed into a discussion of e.g. “is a uniform distribution from 0 to 1 with a range of −10 to 10 the same or different than a uniform distribution from 0 to 1 with a range of 0 to 1”.
This approach is more complex, but sidesteps that.
(Also see https://www.lesswrong.com/posts/mERNQwDNTtqsXbSng/you-can-tell-a-drawing-from-a-painting?commentId=ySCpKgJ8WmN7BFJjN)
What about
pmfA(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩12if x=012if x=20otherwise
pmfB(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩13if x=013if x=113if x=20otherwise ?
Both functions’ support has the same minimum (0) and maximum (2).