I don’t think a less abstract example will solve a dispute over word usage.
What would have been evidence for Uri Geller having psychic powers is if he got more calls than normal people when he did that stunt.
And even if they did receive a statistically significant number of calls, perhaps people lied, grouped together and phoned in supernatural events that hadn’t actually occurred.
If all you have is the one anecdote then it does count as evidence, but only in a strictly philosophical or mathematical sense. Not in any practical sense though since the shift in the relevant p value isn’t going to be visible in the first twenty or thirty decimal places and I doubt anyone alive has that level of precision in their decision-making.
I don’t have a problem with dismissing evidence if it truly is highly irrelevant. I think the problem is that in some cases evidence that at first seems irrelevant turns out to be important.
How relevant a piece of evidence truly is might not become apparent even after significant consideration. This is why, for important matters, I explore as much evidence as possible, even the seemingly irrelevant evidence.
I don’t think a less abstract example will solve a dispute over word usage.
This is really a dispute over maths. The laws of probability are the law, they don’t depend on word usage.
And even if they did receive a statistically significant number of calls, perhaps people lied, grouped together and phoned in supernatural events that hadn’t actually occurred.
Of course people can mock up evidence of things that are not in fact true. It’s still evidence from the perspective of people who are not in on the plot. All of our evidence for everything could in theory have been mocked up by the architects of the Matrix and it would still all be evidence, from our perspective.
I’m not clear how this is relevant to the base rate fallacy though.
How relevant a piece of evidence truly is might not become apparent even after significant consideration. This is why, for important matters, I explore as much evidence as possible, even the seemingly irrelevant evidence.
Strictly speaking what’s going on there is that you are collecting facts which might later turn out to be evidence for a hypothesis you have not articulated yet. This is an entirely rational approach to solving certain classes of problems. It doesn’t change the definition of evidence though,
This is really a dispute over maths. The laws of probability are the law, they don’t depend on word usage.
Explaining the way one uses a word isn’t a statement about maths or the laws of probability either.
I’m not clear how this is relevant to the base rate fallacy though.
It’s not. I was riffing on what you said.
Strictly speaking what’s going on there is that you are collecting facts which might later turn out to be evidence for a hypothesis you have not articulated yet.
I was discussing ascertaining the trustworthiness of evidence concerning a hypothesis you are currently considering. Like an investigation into whether Uri Gellers phone calls were genuine reports of supernatural events, for example.
It doesn’t change the definition of evidence though,
Of course not. I’m not trying to suggest that my usage of “evidence” is somehow better or superior than yours. I do think mine is more common, but that’s a matter of opinion.
Explaining the way one uses a word isn’t a statement about maths or the laws of probability either.
It is, actually. It’s the Bayesian definition that evidence for X is something more likely to be true in a universe where X than in a universe where -X.
Part of the point of Bayes’ Theorem is that correction for the base rate fallacy is baked in. That’s a large part of what separates Bayesian rationality from irrationality,
It is, actually. It’s the Bayesian definition that evidence for X is something more likely to be true in a universe where X than in a universe where -X.
What you’re saying here is that you use Bayes’ theorem to inform your definition of “evidence”.
If I used a different definition of evidence, that doesn’t mean I’m saying something about Bayes’ theorem. That simply means I use the word differently.
When it comes to evidence, I don’t believe Bayes’ theorem deals with the real-world problems that arise when considering a hypothesis. For example, it doesn’t deal with the “garbage in, garbage out” problem.
As I said, we might plug the number of calls Uri Geller got into Bayes’ theorem and because of the answer believe that supernatural events did actually occur. But that would be an incorrect conclusion because we have based our conclusion on the faulty premise that more calls means supernatural events are more likely to have occurred.
What you’re saying here is that you use Bayes’ theorem to inform your definition of “evidence”.
Absolutely. Otherwise I can’t exclude from the domain of “evidence of X” all things which should not incline a rational person to amend their views about X, and I very much want to do that.
Note that something can be “evidence” without being “evidence of X” where X is one specific something. Someone calling in to say their watch stopped is evidence that someone called in and evidence that a watch stopped and so on, just not evidence that Uri Geller has radio-propagated psychic powers.
I agree with you to the extent that Bayes’ theorem is not a magic wand that cures all epistemological ills. You don’t have to browse this site for too long to come across people advancing silly ideas under the banner of Bayesian inference because they are using it incorrectly. However while it’s not a magic wand it’s a mathematical truth about how the universe works, and any time you deliberately deviate from Bayes’ theorem you are deliberately going wrong as a matter of mathematical fact.
As I said, we might plug the number of calls Uri Geller got into Bayes’ theorem and because of the answer believe that supernatural events did actually occur. But that would be an incorrect conclusion because we have based our conclusion on the faulty premise that more calls means supernatural events are more likely to have occurred.
What is actually going on here, assuming you are applying Bayes correctly, is that the prior probability of Uri Geller having radio-propagated psychic powers should be seen as astoundingly low. Far lower than the odds of me winning the lotto twice in a row, for example—let’s say one on ten to the fourteenth as a very generous prior probability. If Uri Geller got a lot more phone calls than a skeptic pretending to be a psychic that should tip the scales a little in his favour but nowhere near enough to get P(Uri Geller has psychic powers) up to a level we should take seriously.
The premise “more calls makes psychic powers more likely” is not flawed at all. If we lived in a bizarre universe where Uri Geller really could reach out through your radio and stop your watch then we would indeed see more calls coming in, and if we lived in that bizarre world we should want to believe we lived in that bizarre world. I’m very sure we do not live in that world but that’s because the evidence is against it, not because I would dismiss that evidence if it supported it.
Absolutely. Otherwise I can’t exclude from the domain of “evidence of X” things which should not incline a rational person to amend their views about X, and I very much want to do that.
If someone believes the Bible is central to question of whether God exists, you can challenge that without having a definition of “evidence” that is informed by Bayes’ theorem.
The premise “more calls makes psychic powers more likely” is not flawed at all.
It could be flawed if there are things that effect the number of phone calls other than Geller’s proposed psychic powers. One show might get more calls but also have more viewers, and that obviously doesn’t make Geller more likely to be psychic during that particular show.
But I am in agreement with you generally with the Uri Geller example. I don’t think phone calls to a television show would alone change my mind, but if we did live in a world where he truly did have psychic powers, I would hope that such evidence would lead me to investigate the matter further.
If someone believes the Bible is central to question of whether God exists, you can challenge that without having a definition of “evidence” that is informed by Bayes’ theorem.
Fair comment. I should have said “all things”, not “things”, and I’ve edited the grandparent appropriately.
It could be flawed if there are things that effect the number of phone calls other than Geller’s proposed psychic powers.
There’s an implicit “all else being equal” in such statements, which really shouldn’t need to be spelled out if all parties are respecting the principle of charity. Any number of things (time of day, stooges working for Uri, number of listeners, service to the telephone system) could also affect the number of calls received, but if all else is equal as far as we are aware then Uri getting more calls than the skeptic is more likely in a universe where he has psychic powers. Hence by Bayes’ theorem it’s evidence he has psychic powers—just extremely weak evidence which is alone insufficient to shift our prior probability much.
It’s also more likely in a universe where Uri has more listeners, has stooges working for him, or has some other non-psychic factor working in his favour of course, and those hypotheses have much, much higher prior probabilities.
When it comes to evidence, I don’t believe Bayes’ theorem deals with the real-world problems that arise when considering a hypothesis. For example, it doesn’t deal with the “garbage in, garbage out” problem.
If you mean things like the base rate fallacy, then yes it does. If you mean that putting in random numbers for your priors doesn’t solve your problems, then there isn’t any method of considering evidence that fixes that in principle.
If you mean things like the base rate fallacy, then yes it does.
In the paragraph after the one you quoted, I gave an example of what I was discussing.
If you mean that putting in random numbers for your priors doesn’t solve your problems, then there isn’t any method of considering evidence that fixes that in principle.
You can check the source of the evidence and try to make sure that you’re not putting in random numbers but reliable data.
When considering hypotheses in the real world—like “Does God exist?” or “Is my wife cheating on me?”—Bayes’ theorem doesn’t encapsulate all the skills you need to arrive at a trustworthy answer. You must clearly understand what it is you’re trying to establish—Aquinas’s conception of God is very different from your average Christian’s. You must be willing to question beliefs that you are attached to or identify with—maybe your wife is sleeping with anything that moves, or maybe you’re a needlessly jealous, insecure husband. You must gather as much evidence as possible, including the evidence that you might initially deem to be irrelevant. You must be diligent, fastidious, and detached when performing the investigation—not hiding behind “Oh, my wife would never do that” or allowing your emotions to effect your judgment.
People will have done all the above and still arrived at erroneous conclusions. Such is the difficulty of testing a hypothesis.
I don’t think a less abstract example will solve a dispute over word usage.
And even if they did receive a statistically significant number of calls, perhaps people lied, grouped together and phoned in supernatural events that hadn’t actually occurred.
I don’t have a problem with dismissing evidence if it truly is highly irrelevant. I think the problem is that in some cases evidence that at first seems irrelevant turns out to be important.
How relevant a piece of evidence truly is might not become apparent even after significant consideration. This is why, for important matters, I explore as much evidence as possible, even the seemingly irrelevant evidence.
This is really a dispute over maths. The laws of probability are the law, they don’t depend on word usage.
Of course people can mock up evidence of things that are not in fact true. It’s still evidence from the perspective of people who are not in on the plot. All of our evidence for everything could in theory have been mocked up by the architects of the Matrix and it would still all be evidence, from our perspective.
I’m not clear how this is relevant to the base rate fallacy though.
Strictly speaking what’s going on there is that you are collecting facts which might later turn out to be evidence for a hypothesis you have not articulated yet. This is an entirely rational approach to solving certain classes of problems. It doesn’t change the definition of evidence though,
Explaining the way one uses a word isn’t a statement about maths or the laws of probability either.
It’s not. I was riffing on what you said.
I was discussing ascertaining the trustworthiness of evidence concerning a hypothesis you are currently considering. Like an investigation into whether Uri Gellers phone calls were genuine reports of supernatural events, for example.
Of course not. I’m not trying to suggest that my usage of “evidence” is somehow better or superior than yours. I do think mine is more common, but that’s a matter of opinion.
It is, actually. It’s the Bayesian definition that evidence for X is something more likely to be true in a universe where X than in a universe where -X.
Part of the point of Bayes’ Theorem is that correction for the base rate fallacy is baked in. That’s a large part of what separates Bayesian rationality from irrationality,
What you’re saying here is that you use Bayes’ theorem to inform your definition of “evidence”.
If I used a different definition of evidence, that doesn’t mean I’m saying something about Bayes’ theorem. That simply means I use the word differently.
When it comes to evidence, I don’t believe Bayes’ theorem deals with the real-world problems that arise when considering a hypothesis. For example, it doesn’t deal with the “garbage in, garbage out” problem.
As I said, we might plug the number of calls Uri Geller got into Bayes’ theorem and because of the answer believe that supernatural events did actually occur. But that would be an incorrect conclusion because we have based our conclusion on the faulty premise that more calls means supernatural events are more likely to have occurred.
Absolutely. Otherwise I can’t exclude from the domain of “evidence of X” all things which should not incline a rational person to amend their views about X, and I very much want to do that.
Note that something can be “evidence” without being “evidence of X” where X is one specific something. Someone calling in to say their watch stopped is evidence that someone called in and evidence that a watch stopped and so on, just not evidence that Uri Geller has radio-propagated psychic powers.
I agree with you to the extent that Bayes’ theorem is not a magic wand that cures all epistemological ills. You don’t have to browse this site for too long to come across people advancing silly ideas under the banner of Bayesian inference because they are using it incorrectly. However while it’s not a magic wand it’s a mathematical truth about how the universe works, and any time you deliberately deviate from Bayes’ theorem you are deliberately going wrong as a matter of mathematical fact.
What is actually going on here, assuming you are applying Bayes correctly, is that the prior probability of Uri Geller having radio-propagated psychic powers should be seen as astoundingly low. Far lower than the odds of me winning the lotto twice in a row, for example—let’s say one on ten to the fourteenth as a very generous prior probability. If Uri Geller got a lot more phone calls than a skeptic pretending to be a psychic that should tip the scales a little in his favour but nowhere near enough to get P(Uri Geller has psychic powers) up to a level we should take seriously.
The premise “more calls makes psychic powers more likely” is not flawed at all. If we lived in a bizarre universe where Uri Geller really could reach out through your radio and stop your watch then we would indeed see more calls coming in, and if we lived in that bizarre world we should want to believe we lived in that bizarre world. I’m very sure we do not live in that world but that’s because the evidence is against it, not because I would dismiss that evidence if it supported it.
If someone believes the Bible is central to question of whether God exists, you can challenge that without having a definition of “evidence” that is informed by Bayes’ theorem.
It could be flawed if there are things that effect the number of phone calls other than Geller’s proposed psychic powers. One show might get more calls but also have more viewers, and that obviously doesn’t make Geller more likely to be psychic during that particular show.
But I am in agreement with you generally with the Uri Geller example. I don’t think phone calls to a television show would alone change my mind, but if we did live in a world where he truly did have psychic powers, I would hope that such evidence would lead me to investigate the matter further.
Fair comment. I should have said “all things”, not “things”, and I’ve edited the grandparent appropriately.
There’s an implicit “all else being equal” in such statements, which really shouldn’t need to be spelled out if all parties are respecting the principle of charity. Any number of things (time of day, stooges working for Uri, number of listeners, service to the telephone system) could also affect the number of calls received, but if all else is equal as far as we are aware then Uri getting more calls than the skeptic is more likely in a universe where he has psychic powers. Hence by Bayes’ theorem it’s evidence he has psychic powers—just extremely weak evidence which is alone insufficient to shift our prior probability much.
It’s also more likely in a universe where Uri has more listeners, has stooges working for him, or has some other non-psychic factor working in his favour of course, and those hypotheses have much, much higher prior probabilities.
If you mean things like the base rate fallacy, then yes it does. If you mean that putting in random numbers for your priors doesn’t solve your problems, then there isn’t any method of considering evidence that fixes that in principle.
In the paragraph after the one you quoted, I gave an example of what I was discussing.
You can check the source of the evidence and try to make sure that you’re not putting in random numbers but reliable data.
When considering hypotheses in the real world—like “Does God exist?” or “Is my wife cheating on me?”—Bayes’ theorem doesn’t encapsulate all the skills you need to arrive at a trustworthy answer. You must clearly understand what it is you’re trying to establish—Aquinas’s conception of God is very different from your average Christian’s. You must be willing to question beliefs that you are attached to or identify with—maybe your wife is sleeping with anything that moves, or maybe you’re a needlessly jealous, insecure husband. You must gather as much evidence as possible, including the evidence that you might initially deem to be irrelevant. You must be diligent, fastidious, and detached when performing the investigation—not hiding behind “Oh, my wife would never do that” or allowing your emotions to effect your judgment.
People will have done all the above and still arrived at erroneous conclusions. Such is the difficulty of testing a hypothesis.