You may want to look at Brandon Fitelson’s short paper Evidence of evidence is not (necessarily) evidence. You seem to be arguing that, since we have strong evidence that the book has strong evidence for Zoroastrianism before we read it, it follows that we already have (the most important part of) our evidence for Zoroastrianism. But it turns out that it’s extremely tricky to make this sort of reasoning work. To use the most primitive example from the paper, discovering that a playing card C is black is evidence that C is the ace of spades. Furthermore, that C is the ace of spades is excellent evidence that it’s an ace. But discovering that C is black does not give you any evidence whatsoever that C is an ace.
The problem here—at least one of them—is that discovering C is black is just as much evidence for C being the x of spades for any other card-value x. Similarly, before opening the book on Zoroastrianism, we have just as much evidence for the existence of strong evidence for Christianity/atheism/etc, so our credences shouldn’t suddenly start favoring any one of these. But once we learn the evidence for Zoroastrianism, we’ve acquired new information, in just the same way that learning that the card is an ace of spades provides us new information if we previously just knew it was black.
I do suspect that there are relevant disanalogies here, but don’t have a very detailed understanding of them.
You seem to be arguing that, since we have strong evidence that the book has strong evidence for Zoroastrianism before we read it, it follows that we already have (the most important part of) our evidence for Zoroastrianism.
Not exactly. I do think this would be a true statement, if the book was a genuine book on Zoroastrianism and not a book which we know was designed to deceive us. But as far as I know it’s only tangentially connected to the argument I’m making.
You may want to look at Brandon Fitelson’s short paper Evidence of evidence is not (necessarily) evidence.
Thanks for summarizing the paper; I tried to read it but it was written in a way that seemed designed to be as obscure as possible. Your explanation makes more sense.
But I still don’t see the problem. Learning a card is black increases the chance it’s the ace of spades or clubs, but decreases the chance it’s the ace of hearts or diamonds. The chance that it’s the ace of spades becomes greater, but the net chance that it’s an ace remains exactly the same. Evidence of evidence is still evidence, but evidence of evidence plus evidence of evidence that goes the opposite direction cancel out and make zero evidence.
Again, I’m not sure about the relevance here. It’s not the case that, merely by knowing the book exists without reading it, we have new evidence for the existence of some evidence which both supports and, in a different way, opposes Zoroastrianism.
Thanks for summarizing the paper; I tried to read it but it was written in a way that seemed designed to be as obscure as possible. Your explanation makes more sense.
(I guess I’d say it was written in a way designed to be precise. But I agree that the author isn’t the best writer.)
Evidence of evidence is still evidence, but evidence of evidence plus evidence of evidence that goes the opposite direction cancel out and make zero evidence.
I find this sentence hard to make sense of. Based on the first part of the sentence, you seem to be suggesting that the problem in the card scenario is that our evidence^2 (= the card is black) is both evidence^1 for the card being an ace and evidence^1 against the card being an ace, and the two pieces of evidence^1 balance out to yield the same total probability of the card being an ace as before. But clearly no single piece of information, such as the card being black, can provide evidence^1 both for and against a given hypothesis. It either yields evidence^1 or it doesn’t. And if it doesn’t, then evidence^2 is not always evidence^1.
Anyway, the relevance is this: When we learn the card is black, we acquire evidence for a bunch of different pieces of information which, taken on their own, have varying probabilistic effects on the hypothesis that the card is an ace. These effects add up in such a way as to leave the posterior probability of the hypothesis untouched. But once we actually learn one of these individual pieces of information, suddenly the posterior shoots way up.
Similarly, before we read the book, we have evidence for a bunch of different pieces of information which, taken on their own, have varying probabilistic effects on the truth of Zoroastrianism. These effects add up in such a way as to leave our posterior in Zoroastrianism untouched (assuming we don’t consider non-book-possessing religions). So why is it that when we learn one of these pieces of information by reading the book, our posterior shouldn’t change, unlike in the card case?
O.K., here’s a disanalogy that may be important. In the card case, learning that C is the ace of spades should drastically lower our credence of the card being another x of spades. On the other hand, after reading the Zoroastrianism book, we shouldn’t significantly doubt that the other books contain strong evidence, as well, given the known capabilities of the AI.
You may want to look at Brandon Fitelson’s short paper Evidence of evidence is not (necessarily) evidence. You seem to be arguing that, since we have strong evidence that the book has strong evidence for Zoroastrianism before we read it, it follows that we already have (the most important part of) our evidence for Zoroastrianism. But it turns out that it’s extremely tricky to make this sort of reasoning work. To use the most primitive example from the paper, discovering that a playing card C is black is evidence that C is the ace of spades. Furthermore, that C is the ace of spades is excellent evidence that it’s an ace. But discovering that C is black does not give you any evidence whatsoever that C is an ace.
The problem here—at least one of them—is that discovering C is black is just as much evidence for C being the x of spades for any other card-value x. Similarly, before opening the book on Zoroastrianism, we have just as much evidence for the existence of strong evidence for Christianity/atheism/etc, so our credences shouldn’t suddenly start favoring any one of these. But once we learn the evidence for Zoroastrianism, we’ve acquired new information, in just the same way that learning that the card is an ace of spades provides us new information if we previously just knew it was black.
I do suspect that there are relevant disanalogies here, but don’t have a very detailed understanding of them.
Not exactly. I do think this would be a true statement, if the book was a genuine book on Zoroastrianism and not a book which we know was designed to deceive us. But as far as I know it’s only tangentially connected to the argument I’m making.
Thanks for summarizing the paper; I tried to read it but it was written in a way that seemed designed to be as obscure as possible. Your explanation makes more sense.
But I still don’t see the problem. Learning a card is black increases the chance it’s the ace of spades or clubs, but decreases the chance it’s the ace of hearts or diamonds. The chance that it’s the ace of spades becomes greater, but the net chance that it’s an ace remains exactly the same. Evidence of evidence is still evidence, but evidence of evidence plus evidence of evidence that goes the opposite direction cancel out and make zero evidence.
Again, I’m not sure about the relevance here. It’s not the case that, merely by knowing the book exists without reading it, we have new evidence for the existence of some evidence which both supports and, in a different way, opposes Zoroastrianism.
(I guess I’d say it was written in a way designed to be precise. But I agree that the author isn’t the best writer.)
I find this sentence hard to make sense of. Based on the first part of the sentence, you seem to be suggesting that the problem in the card scenario is that our evidence^2 (= the card is black) is both evidence^1 for the card being an ace and evidence^1 against the card being an ace, and the two pieces of evidence^1 balance out to yield the same total probability of the card being an ace as before. But clearly no single piece of information, such as the card being black, can provide evidence^1 both for and against a given hypothesis. It either yields evidence^1 or it doesn’t. And if it doesn’t, then evidence^2 is not always evidence^1.
Anyway, the relevance is this: When we learn the card is black, we acquire evidence for a bunch of different pieces of information which, taken on their own, have varying probabilistic effects on the hypothesis that the card is an ace. These effects add up in such a way as to leave the posterior probability of the hypothesis untouched. But once we actually learn one of these individual pieces of information, suddenly the posterior shoots way up.
Similarly, before we read the book, we have evidence for a bunch of different pieces of information which, taken on their own, have varying probabilistic effects on the truth of Zoroastrianism. These effects add up in such a way as to leave our posterior in Zoroastrianism untouched (assuming we don’t consider non-book-possessing religions). So why is it that when we learn one of these pieces of information by reading the book, our posterior shouldn’t change, unlike in the card case?
Thank you, that’s what I was trying to get at, but didn’t know how.
O.K., here’s a disanalogy that may be important. In the card case, learning that C is the ace of spades should drastically lower our credence of the card being another x of spades. On the other hand, after reading the Zoroastrianism book, we shouldn’t significantly doubt that the other books contain strong evidence, as well, given the known capabilities of the AI.
This isn’t a very formal treatment, though.
“Evidence of evidence is more likely to be filtered evidence” is a more accurate phrasing.
I’m not exactly sure what the “more likely” here means. More likely than what?
The link keeps omitting the colon in the “http://.” I don’t know why it’s doing that.
Evidence of evidence is not (necessarily) evidence
Markup code: