Due to low karma i can’t reply in the relevant comment thread, so I do it here:
Desrtopa wrote:
If you think there’s an inconsistency in people’s dismissal of UFO eyewitness accounts, I think you may not have grasped the principles of Bayesian reasoning yet.
Baysian reasoning says to treat all evidence equal. The prior may be low, say 0.01, but that’s all. There is no weight of the prior that can change. Thus if we see new evidence the update of the probability isn’t affected by some magic weight that could diminuize the size of the update arbitrarily much. Agree?
All the ‘lowness’ of the prior does is to ensure that the posterior doesn’t get updated quite as much as it would with a higher prior. I.e. the effect of the prior scales linearly. Agree? Thus, people in this thread saying “eye witness accounts does not count as evidence when it comes to ufos”* can’t account for this decision in a Bayesian manner. Agree?
more precisely they are saying: “seeing objects that can’t be explained with natural or man made phenomena does not count as evidence for there actually being such objects”. This way of thinking has nothing in common with the Bayesian approach. Agree?
You also stated that eye witness accounts of rare things is less reliable than of common things and thus should count less. Is this really so? Isn’t a low prior the only mean for the bayesian method to affect the effect of evidence?
If I see a gremlin in plain sight, or rabbit plain as light, Bayesian reasoning wouldn’t distinguish between these reports solely on their own account.
The idea that eye witness accounts of rare things is less reliable than eye witness accounts of common things must be grounded in something else than Bayesian reasoning. Perhaps in psychology. Thus I can ask you: do you have a reference to back up your proposition?
Also I can ask you: Do you find your proposition to be in good correspondence with this statement from the Project Blue Book Special Report No. 14: “The higher the quality of the case, the more likely it was to be classified unknown. 35% of the excellent cases were deemed unknowns, as opposed to only 18% of the poorest cases. This was the exact opposite of the result predicted by skeptics, who usually argued unknowns were poorer quality cases involving unreliable witnesses that could be solved if only better information were available.” ?
If you think there’s an inconsistency in people’s dismissal of UFO eyewitness accounts, I think you may not have grasped the principles of Bayesian reasoning yet.
Eliezer’s introduction is here, but to add some subject relevant commentary:
Eyewitness reports of unfamiliar things tend to be much less reliable than eyewitness reports of familiar ones. If a person witnesses someone they already recognize committing a crime, for instance, their description of the perpetrator is likely to be highly accurate, whereas a person who witnesses someone they don’t recognize committing a crime is likely to give a description that’s extremely unhelpful, and often worse than useless.
So as a general principle, people reporting on unfamiliar phenomena tends to be weaker evidence than people reporting on familiar ones. If your girlfriend says she saw an eagle by the creek, and she often sees eagles and is well equipped to recognize them on sight, then her report is stronger evidence than that of a tourist who thinks they know what eagles look like, and saw a bird which fit their idea of what an eagle is supposed to look like, so they’re pretty sure that’s what it was, which is in turn stronger evidence than a report from someone who doesn’t know what eagles look like at all, but saw a bird which they think fit the description of an eagle which they heard after seeing it.
This is in addition to the fact that the single standard of Bayesian evidential reasoning demands larger amounts of evidence to raise less probable events to the point of likelihood. If seeing an eagle over the creek has a prior of .02, your girlfriend’s say-so is more than enough evidence to accept the proposition, unless she’s unusually dishonest. If she claims to have seen a gremlin, on the other hand, the prior is going to be much, much lower; if gremlins exist at all, they must be awfully rare and elusive to have avoided reliable observation thus far. So the likelihood that the report is due to misidentification, dishonesty, confusion, or some other reason for a false report, swamps the likelihood that she really saw a gremlin, under the same standards of evidence as we would apply in the case of the eagle.
Due to low karma i can’t reply in the relevant comment thread, so I do it here:
Desrtopa wrote:
Baysian reasoning says to treat all evidence equal. The prior may be low, say 0.01, but that’s all. There is no weight of the prior that can change. Thus if we see new evidence the update of the probability isn’t affected by some magic weight that could diminuize the size of the update arbitrarily much. Agree?
All the ‘lowness’ of the prior does is to ensure that the posterior doesn’t get updated quite as much as it would with a higher prior. I.e. the effect of the prior scales linearly. Agree? Thus, people in this thread saying “eye witness accounts does not count as evidence when it comes to ufos”* can’t account for this decision in a Bayesian manner. Agree?
more precisely they are saying: “seeing objects that can’t be explained with natural or man made phenomena does not count as evidence for there actually being such objects”. This way of thinking has nothing in common with the Bayesian approach. Agree?
You also stated that eye witness accounts of rare things is less reliable than of common things and thus should count less. Is this really so? Isn’t a low prior the only mean for the bayesian method to affect the effect of evidence?
If I see a gremlin in plain sight, or rabbit plain as light, Bayesian reasoning wouldn’t distinguish between these reports solely on their own account.
The idea that eye witness accounts of rare things is less reliable than eye witness accounts of common things must be grounded in something else than Bayesian reasoning. Perhaps in psychology. Thus I can ask you: do you have a reference to back up your proposition?
Also I can ask you: Do you find your proposition to be in good correspondence with this statement from the Project Blue Book Special Report No. 14: “The higher the quality of the case, the more likely it was to be classified unknown. 35% of the excellent cases were deemed unknowns, as opposed to only 18% of the poorest cases. This was the exact opposite of the result predicted by skeptics, who usually argued unknowns were poorer quality cases involving unreliable witnesses that could be solved if only better information were available.” ?
This is the original comment from Desrtopa:
If you think there’s an inconsistency in people’s dismissal of UFO eyewitness accounts, I think you may not have grasped the principles of Bayesian reasoning yet.
Eliezer’s introduction is here, but to add some subject relevant commentary:
Eyewitness reports of unfamiliar things tend to be much less reliable than eyewitness reports of familiar ones. If a person witnesses someone they already recognize committing a crime, for instance, their description of the perpetrator is likely to be highly accurate, whereas a person who witnesses someone they don’t recognize committing a crime is likely to give a description that’s extremely unhelpful, and often worse than useless.
So as a general principle, people reporting on unfamiliar phenomena tends to be weaker evidence than people reporting on familiar ones. If your girlfriend says she saw an eagle by the creek, and she often sees eagles and is well equipped to recognize them on sight, then her report is stronger evidence than that of a tourist who thinks they know what eagles look like, and saw a bird which fit their idea of what an eagle is supposed to look like, so they’re pretty sure that’s what it was, which is in turn stronger evidence than a report from someone who doesn’t know what eagles look like at all, but saw a bird which they think fit the description of an eagle which they heard after seeing it.
This is in addition to the fact that the single standard of Bayesian evidential reasoning demands larger amounts of evidence to raise less probable events to the point of likelihood. If seeing an eagle over the creek has a prior of .02, your girlfriend’s say-so is more than enough evidence to accept the proposition, unless she’s unusually dishonest. If she claims to have seen a gremlin, on the other hand, the prior is going to be much, much lower; if gremlins exist at all, they must be awfully rare and elusive to have avoided reliable observation thus far. So the likelihood that the report is due to misidentification, dishonesty, confusion, or some other reason for a false report, swamps the likelihood that she really saw a gremlin, under the same standards of evidence as we would apply in the case of the eagle.
In short: Desrtopa was right and I was wrong: all evidence is not equal.