(I can’t be bothered to figure out the correct constants to normalize these. Should be fairly straightforward to calculate from the standard infinite-geometric-series-sum formula.)
Both of these functions have the same range, −1<x<1, but there is no overlap between the two functions. Their support is disjoint.
They are both (un-normalized) probability density functions, as per the names pdfA and pdfB. My apologies if that was unclear.
To be somewhat clearer: I was referring to the probability distributions described by these two probability density functions. They have the same range, but disjoint support, and so anything you drew from B could not have been drawn from A.
In other words, a (pathological) counterexample to “The range of the distributions is the same, so anything you draw from B could have been drawn from A”.
In mathematics, the range of a function may refer to either of two closely related concepts: The codomain of the function; The image of the function.
I meant the image. At least that’s what you call it for a function; I don’t know the terminology for distributions. Honestly I wasn’t thinking much about the word “range”, and should have simply said:
Anything you draw from B could have been drawn from A. And yet...
Before anyone starts on about how this statement isn’t well defined because the probability that you select any particular value from a continuous distribution, I’ll point out that I’ve never seen anyone draw a real number uniformly at random between 0 and 1 from a hat. Even if you are actually selecting from a continuous distribution, the observations we can make about it are finite, so the relevant probabilities are all finite.
I was assuming you meant range as in the statistical term (for a distribution, roughly, the maximum x for which pdf(x)≠0, minus the minimum x for which pdf(x)≠0).
Annoyingly, this is closer to the domain than it is the range, in function terminology.
I meant the image.
Are you sure? The range is a description of the possible outputs of the pdf, which means… almost nothing. Trivial counterexample if you do mean image:
Uniform distribution A between 0 and 0.5 (that is, 2 for 0..0.5, and 0 otherwise).
Uniform distribution B between 1.0 and 1.5 (that is, 2 for 1.0..1.5, and 0 otherwise).
Both of these distributions have the same image {0, 2}. And yet they are disjoint.
Honestly I wasn’t thinking much about the word “range”, and should have simply said: > Anything you draw from B could have been drawn from A.
There are many probability distributions where this is not the case. (Like the two uniform distributions A and B I give in this post.)
*****
Oh. You said you don’t know the terminology for distributions. Is it possible you’re under a misunderstanding of what a distribution is? It’s an “input” of a possible result, and an “output” of how probable that result is[1]. The output is not a result. The input is.
Oh. You said you don’t know the terminology for distributions. Is it possible you’re under a misunderstanding of what a distribution is? It’s an “input” of a possible result, and an “output” of how probable that result is.
Yup, it was that. I thought “possible values of the distribution”, and my brain output “range, like in functions”. I shall endeavor not to use a technical term when I don’t mean it or need it, because wow was this a tangent.
If I may ask, why didn’t you use the following (simpler imo) example: pmf_A(0) = 1 pmf_A(1) = 0 pmf_B(0) = 0 pmf_B(1) = 1
With that approach one can argue that the two PMFs have different ranges[1], and get rabbit-holed into a discussion of e.g. “is a uniform distribution from 0 to 1 with a range of −10 to 10 the same or different than a uniform distribution from 0 to 1 with a range of 0 to 1”.
This approach is more complex, but sidesteps that.
This does not hold in pathological cases, I don’t think. As one example:
pdfA(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩...0,−1+1/24≤x<−1+1/231,−1+1/23≤x<−1+1/220,−1+1/22≤x<−1+1/211,−1+1/21≤x<1−1/210,1−1/21≤x<1−1/221,1−1/22≤x<1−1/230,1−1/23≤x<1−1/24...
pdfB(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩...1,−1+1/24≤x<−1+1/230,−1+1/23≤x<−1+1/221,−1+1/22≤x<−1+1/210,−1+1/21≤x<1−1/211,1−1/21≤x<1−1/220,1−1/22≤x<1−1/231,1−1/23≤x<1−1/24...
(I can’t be bothered to figure out the correct constants to normalize these. Should be fairly straightforward to calculate from the standard infinite-geometric-series-sum formula.)
Both of these functions have the same range, −1<x<1, but there is no overlap between the two functions. Their support is disjoint.
That’s a function, he was referring to a distribution
They are both (un-normalized) probability density functions, as per the names pdfA and pdfB. My apologies if that was unclear.
To be somewhat clearer: I was referring to the probability distributions described by these two probability density functions. They have the same range, but disjoint support, and so anything you drew from B could not have been drawn from A.
In other words, a (pathological) counterexample to “The range of the distributions is the same, so anything you draw from B could have been drawn from A”.
Wikipedia says:
I meant the image. At least that’s what you call it for a function; I don’t know the terminology for distributions. Honestly I wasn’t thinking much about the word “range”, and should have simply said:
Before anyone starts on about how this statement isn’t well defined because the probability that you select any particular value from a continuous distribution, I’ll point out that I’ve never seen anyone draw a real number uniformly at random between 0 and 1 from a hat. Even if you are actually selecting from a continuous distribution, the observations we can make about it are finite, so the relevant probabilities are all finite.
I was assuming you meant range as in the statistical term (for a distribution, roughly, the maximum x for which pdf(x)≠0, minus the minimum x for which pdf(x)≠0).
Annoyingly, this is closer to the domain than it is the range, in function terminology.
Are you sure? The range is a description of the possible outputs of the pdf, which means… almost nothing. Trivial counterexample if you do mean image:
Uniform distribution A between 0 and 0.5 (that is, 2 for 0..0.5, and 0 otherwise).
Uniform distribution B between 1.0 and 1.5 (that is, 2 for 1.0..1.5, and 0 otherwise).
Both of these distributions have the same image {0, 2}. And yet they are disjoint.
There are many probability distributions where this is not the case. (Like the two uniform distributions A and B I give in this post.)
*****
Oh. You said you don’t know the terminology for distributions. Is it possible you’re under a misunderstanding of what a distribution is? It’s an “input” of a possible result, and an “output” of how probable that result is[1]. The output is not a result. The input is.
...to way oversimplify, especially for continuous distributions.
Yup, it was that. I thought “possible values of the distribution”, and my brain output “range, like in functions”. I shall endeavor not to use a technical term when I don’t mean it or need it, because wow was this a tangent.
If I may ask, why didn’t you use the following (simpler imo) example: pmf_A(0) = 1 pmf_A(1) = 0 pmf_B(0) = 0 pmf_B(1) = 1
Or even the “Bit sequences” part of the post?
With that approach one can argue that the two PMFs have different ranges[1], and get rabbit-holed into a discussion of e.g. “is a uniform distribution from 0 to 1 with a range of −10 to 10 the same or different than a uniform distribution from 0 to 1 with a range of 0 to 1”.
This approach is more complex, but sidesteps that.
(Also see https://www.lesswrong.com/posts/mERNQwDNTtqsXbSng/you-can-tell-a-drawing-from-a-painting?commentId=ySCpKgJ8WmN7BFJjN)
What about
pmfA(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩12if x=012if x=20otherwise
pmfB(x)=⎧⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎨⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪⎩13if x=013if x=113if x=20otherwise ?
Both functions’ support has the same minimum (0) and maximum (2).