Going outside the argument, if you come up with 50% chance that any given observation of a UAP was actually observing a UAP, you’ve done something wrong.
This percentage is conditional on P( observation | UAP ) being high (0.8). I guess only a minority of such observations (if any) are that high.
Didn’t click through to the study, but what is an ‘observation’ in this case? I make 3200 observations a day, but I don’t observe 70 UAPs a day. It seems far more likely that this was 3200 instances of someone reporting a UAP—in other words, the 22% is pointing towards P(UAP|UO), not P(UAP). But “we couldn’t identify this” doesn’t necessarily mean it was a UAP, so this is still an overestimate.
If this is so, then I can’t use any empirical study to estimate P( UAP ). What value should I then guesstimate P( UAP ) to be? If I choose a generic 0.5, then the result will be an even higher P( UAP | observation ) than 0.53. Thus it would be even harder to dismiss the observation.
If I instead make my own estimation, say, 0.0000001, then one could ask if this was still an epistemologically sound approach. My choice would be rather arbitrary.
Previously we decided P(UO1|UAP) to be 0.8 [...] Thus P(UO1|¬UAP) must be 1−0.8=0.2
If you don’t have an epistemically sound approach, then you should probably say “I don’t know” instead of using an epistemically unsound one, or at least say “this is really bad, and you shouldn’t put high confidence in my conclusions, but it’s the best I can do, so...”
That said, one option you have is not to calculate P(UAP) at all, and instead calculate a likelihood ratio:
So if you just calculate P(UO1|UAP) / P(UO1|¬UAP), then anyone can update their P(UAP) appropriately, regardless of where it started.
Could you elaborate?
P(rain | clouds) might be something like 0.7, and that means that P(¬rain | clouds) is 0.3. But P(rain | ¬clouds) is 0.
You simply can’t calculate P(UO1|¬UAP) from P(UO1|UAP). You need to work it out some other way.
I also don’t think that asking for P(UO1|UAP) and P(UO1|¬UAP) is reasonable without knowing anything about UO1. Right now I’m observing my watch tick; that’s no more or less likely to happen in UAP-world than ¬UAP-world, so the likelihood ratio is one. If tomorrow night I go outside and see lots of bright lights in the sky, and a crop circle the next morning (which is especially weird because there didn’t used to be any crops there at all), and the news reports that lots of other people have seen the same thing and the government is passing it off as a sighting of Venus, then that’s somewhat more likely in UAP-world than ¬UAP-world.
If tomorrow night I go outside and see lots of bright lights in the sky, and a crop circle the next morning (which is especially weird because there didn’t used to be any crops there at all), and the news reports that lots of other people have seen the same thing and the government is passing it off as a sighting of Venus, then that’s somewhat more likely in UAP-world than ¬UAP-world.
This example suggests that you’re confusing P(OU1|UAP) with P(UAP|OU1). To determine P(OU1|UAP), image you live in a world where UAP is true.
Unfortunately, the analysis so far hasn’t been clear on what we mean by P(UAP): does it mean the probability that there are alien visitors within Earth’s atmosphere or that their ship is flying over head right now?
Assuming the former, to estimate P(OU1|UAP) assume there are aliens on Earth, if that is the case what’s the probability of you observing the light in the sky. Obviously this is hard to estimate but one would start by speculating about the potential motives and behavior of the aliens.
I’m afraid I can’t tell which direction you think I’m confused in. That example was intended to be an instance of UO1 for which P(UO1|UAP) > P(UO1|¬UAP), and that still seems true to me, even if P(UO1|UAP) is still low.
(I’m taking UAP to be something like “Earth is sometimes visited by aliens”.)
If you don’t have an epistemically sound approach, then you should probably say “I don’t know” instead of using an epistemically unsound one, or at least say “this is really bad, and you shouldn’t put high confidence in my conclusions, but it’s the best I can do, so...”
Instead of relying on dubious priors couldn’t one simply avoid having to reliably estimate a prior probability P(UAP) by choosing a canonical dataset of observations, choosing a generic prior P(UAP) = 0.5 and then repeatedly update P(UAP | observation x) for each observation x in the dataset?
In this way, the unreliable prior should gradually be deluted, through the iterations. In the end, it will be overshadowed by the influence of the canonical observation data.
If so, how could one do this programmatically?
And how could one do this analytically?
(links are welcome!)
I also hinted at these options in the section ‘Future work’ in the article. But I don’t know how to approach this approach..
I also don’t think that asking for P(UO1|UAP) and P(UO1|¬UAP) is reasonable without knowing anything about UO1. Right now I’m observing my watch tick; that’s no more or less likely to happen in UAP-world than ¬UAP-world, so the likelihood ratio is one. If tomorrow night I go outside and see lots of bright lights in the sky, and a crop circle the next morning (which is especially weird because there didn’t used to be any crops there at all), and the news reports that lots of other people have seen the same thing and the government is passing it off as a sighting of Venus, then that’s somewhat more likely in UAP-world than ¬UAP-world.
As the goal is to say something prior to investigating the observation, I must assume as little as possible about the nature of the given observation.
In the article I assumed P ( observation | UAP ) to be 0.8.
If I could reuse this bit of information to say something about P(UO1|UAP) and P(UO1|¬UAP), then I haven’t broken the “let’s assume as little as possibly”-premise any further.
Is that bit of information sufficient to say something useful about P(UO1|UAP) and P(UO1|¬UAP)?
canonical dataset of observations [...] unreliable prior should gradually be diluted
Indeed, if you have enough observations then the prior eventually doesn’t matter. The difficulty is in the selection of the observations. Ideally you should include every potentially relevant observation—including, e.g., every time someone looks up at the sky and doesn’t see an alien spaceship, and every time anyone operates a radar or a radio telescope or whatever and sees nothing out of the ordinary.
In practice it’s simply impractical to incorporate every potentially relevant observation into our thinking. But that makes it awfully easy to have some bias in selection, and that can make a huge difference to the conclusions.
In practice it’s simply impractical to incorporate every potentially relevant observation into our thinking. But that makes it awfully easy to have some bias in selection, and that can make a huge difference to the conclusions.
Yes these circumstances induce bias and this is unfortunate if one wants to say anything about frequency and such things.
Another somewhat simpler question is this: given n observations of something the observer thinks is a UAP, what is the probability that at least one of these observations originated from a UAP?
If for each of these observations P( observation | UAP ) is strictly greater than 0, then I suspect P(UAP) will go towards 1, monotonously, as the number of observations increases.
Is this hunch somewhat correct? How do I express this hunch mathematically..?
I also touch on this question in the section ‘Future work’ in my article, but I don’t have the answer.
If for each of these observations P( observation | UAP ) is strictly greater than 0, then I suspect P(UAP) will go towards 1, monotonously, as the number of observations increases.
No. This violates the law of conservation of expected evidence. The relevant question is whether P( observation | UAP ) is bigger or smaller than P( observation | ~UAP ).
The problem, as I mentioned above, is that it’s hard to estimate P( observation | UAP ).
What if we have n observations where P( observation | ~UAP ) through investigation has been found to be 0 and, while hard to determine, P( observation | UAP ) is reasonably said to be strictly greater than 0.
Then P(UAP) will go towards 1, monotonously, as the number of observations increases, right?
What if we have n observations where P( observation | ~UAP ) through investigation has been found to be 0
Um, I don’t think you understand what it means for P( observation | ~UAP ) to equal 0. If P( observation | ~UAP ) were really 0, then a single such observation would be enough to comclude the P(UAP) is 1.
So how should one interpret findings like this:
“We investigated n observations and out of these there were k observations which had sufficient observation data to rule out all known aerial phenomena as being the cause”.
So that would imply that P(UAP) is pretty much 1?
So what remains is “merely” to determine what lies in this set ‘UAP’ as it could pretty much be anything.
So how should one interpret findings like this: “We investigated n observations and out of these there were k observations which had sufficient observation data to rule out all known aerial phenomena as being the cause”.
If I take that statement at face value it means the observations were caused by some unknown phenomenon. Therefore, unknown phenomena of this type exist.
What value should I then guesstimate P( UAP ) to be? If I choose a generic 0.5, then the result will be an even higher P( UAP | observation ) than 0.53. Thus it would be even harder to dismiss the observation.
When I think of it, I recall I’m not the only one trying to estimate P(UAP). The Fermi paradox concludes something like this:
“Aliens surely do exist, so how come we don’t see them?”
In this sentence lies an estimate of P(UAP). The only problem is that it is rather high ( ‘surely’ == 0.9 ?), thus making it even harder again to dismiss P( UAP | observation )
That isn’t an estimate of P(UAP). The following two propositions are very different:
“There are technologically advanced civilizations elsewhere in the universe.”
“Sometimes things happen visibly in our skies that aren’t adequately explained by anything uncontroversially known to be real.”
The first of these can be true without the second—maybe there are aliens but they never come here. (That was rather the point of the Fermi paradox.)
The second can be true without the first—maybe people have actually seen angels, or time travellers from earth’s future, or spontaneous rips in the fabric of spacetime.
This percentage is conditional on P( observation | UAP ) being high (0.8). I guess only a minority of such observations (if any) are that high.
If this is so, then I can’t use any empirical study to estimate P( UAP ). What value should I then guesstimate P( UAP ) to be? If I choose a generic 0.5, then the result will be an even higher P( UAP | observation ) than 0.53. Thus it would be even harder to dismiss the observation.
If I instead make my own estimation, say, 0.0000001, then one could ask if this was still an epistemologically sound approach. My choice would be rather arbitrary.
Could you elaborate?
Thanks for your feedback :)
If you don’t have an epistemically sound approach, then you should probably say “I don’t know” instead of using an epistemically unsound one, or at least say “this is really bad, and you shouldn’t put high confidence in my conclusions, but it’s the best I can do, so...”
That said, one option you have is not to calculate P(UAP) at all, and instead calculate a likelihood ratio:
P(UAP|UO1) / P(¬UAP|UO1) = P(UO1|UAP) / P(UO1|¬UAP) × P(UAP)/P(¬UAP)
So if you just calculate P(UO1|UAP) / P(UO1|¬UAP), then anyone can update their P(UAP) appropriately, regardless of where it started.
P(rain | clouds) might be something like 0.7, and that means that P(¬rain | clouds) is 0.3. But P(rain | ¬clouds) is 0.
You simply can’t calculate P(UO1|¬UAP) from P(UO1|UAP). You need to work it out some other way.
I also don’t think that asking for P(UO1|UAP) and P(UO1|¬UAP) is reasonable without knowing anything about UO1. Right now I’m observing my watch tick; that’s no more or less likely to happen in UAP-world than ¬UAP-world, so the likelihood ratio is one. If tomorrow night I go outside and see lots of bright lights in the sky, and a crop circle the next morning (which is especially weird because there didn’t used to be any crops there at all), and the news reports that lots of other people have seen the same thing and the government is passing it off as a sighting of Venus, then that’s somewhat more likely in UAP-world than ¬UAP-world.
This example suggests that you’re confusing P(OU1|UAP) with P(UAP|OU1). To determine P(OU1|UAP), image you live in a world where UAP is true.
Unfortunately, the analysis so far hasn’t been clear on what we mean by P(UAP): does it mean the probability that there are alien visitors within Earth’s atmosphere or that their ship is flying over head right now?
Assuming the former, to estimate P(OU1|UAP) assume there are aliens on Earth, if that is the case what’s the probability of you observing the light in the sky. Obviously this is hard to estimate but one would start by speculating about the potential motives and behavior of the aliens.
I’m afraid I can’t tell which direction you think I’m confused in. That example was intended to be an instance of UO1 for which P(UO1|UAP) > P(UO1|¬UAP), and that still seems true to me, even if P(UO1|UAP) is still low.
(I’m taking UAP to be something like “Earth is sometimes visited by aliens”.)
Instead of relying on dubious priors couldn’t one simply avoid having to reliably estimate a prior probability P(UAP) by choosing a canonical dataset of observations, choosing a generic prior P(UAP) = 0.5 and then repeatedly update P(UAP | observation x) for each observation x in the dataset?
In this way, the unreliable prior should gradually be deluted, through the iterations. In the end, it will be overshadowed by the influence of the canonical observation data.
If so, how could one do this programmatically? And how could one do this analytically? (links are welcome!)
I also hinted at these options in the section ‘Future work’ in the article. But I don’t know how to approach this approach..
As the goal is to say something prior to investigating the observation, I must assume as little as possible about the nature of the given observation. In the article I assumed P ( observation | UAP ) to be 0.8.
If I could reuse this bit of information to say something about P(UO1|UAP) and P(UO1|¬UAP), then I haven’t broken the “let’s assume as little as possibly”-premise any further.
Is that bit of information sufficient to say something useful about P(UO1|UAP) and P(UO1|¬UAP)?
Indeed, if you have enough observations then the prior eventually doesn’t matter. The difficulty is in the selection of the observations. Ideally you should include every potentially relevant observation—including, e.g., every time someone looks up at the sky and doesn’t see an alien spaceship, and every time anyone operates a radar or a radio telescope or whatever and sees nothing out of the ordinary.
In practice it’s simply impractical to incorporate every potentially relevant observation into our thinking. But that makes it awfully easy to have some bias in selection, and that can make a huge difference to the conclusions.
Yes these circumstances induce bias and this is unfortunate if one wants to say anything about frequency and such things.
Another somewhat simpler question is this: given n observations of something the observer thinks is a UAP, what is the probability that at least one of these observations originated from a UAP?
If for each of these observations P( observation | UAP ) is strictly greater than 0, then I suspect P(UAP) will go towards 1, monotonously, as the number of observations increases.
Is this hunch somewhat correct? How do I express this hunch mathematically..?
I also touch on this question in the section ‘Future work’ in my article, but I don’t have the answer.
http://myinnerouterworldsimulator.neocities.org/index.html
No. This violates the law of conservation of expected evidence. The relevant question is whether P( observation | UAP ) is bigger or smaller than P( observation | ~UAP ).
The problem, as I mentioned above, is that it’s hard to estimate P( observation | UAP ).
What if we have n observations where P( observation | ~UAP ) through investigation has been found to be 0 and, while hard to determine, P( observation | UAP ) is reasonably said to be strictly greater than 0.
Then P(UAP) will go towards 1, monotonously, as the number of observations increases, right?
Um, I don’t think you understand what it means for P( observation | ~UAP ) to equal 0. If P( observation | ~UAP ) were really 0, then a single such observation would be enough to comclude the P(UAP) is 1.
So how should one interpret findings like this: “We investigated n observations and out of these there were k observations which had sufficient observation data to rule out all known aerial phenomena as being the cause”.
So that would imply that P(UAP) is pretty much 1?
So what remains is “merely” to determine what lies in this set ‘UAP’ as it could pretty much be anything.
If I take that statement at face value it means the observations were caused by some unknown phenomenon. Therefore, unknown phenomena of this type exist.
When I think of it, I recall I’m not the only one trying to estimate P(UAP). The Fermi paradox concludes something like this:
“Aliens surely do exist, so how come we don’t see them?”
In this sentence lies an estimate of P(UAP). The only problem is that it is rather high ( ‘surely’ == 0.9 ?), thus making it even harder again to dismiss P( UAP | observation )
http://en.wikipedia.org/wiki/Fermi_paradox
That isn’t an estimate of P(UAP). The following two propositions are very different:
“There are technologically advanced civilizations elsewhere in the universe.”
“Sometimes things happen visibly in our skies that aren’t adequately explained by anything uncontroversially known to be real.”
The first of these can be true without the second—maybe there are aliens but they never come here. (That was rather the point of the Fermi paradox.)
The second can be true without the first—maybe people have actually seen angels, or time travellers from earth’s future, or spontaneous rips in the fabric of spacetime.