Vladimir, Cyan, and jimrandomh, since you essentially said the same thing, consider this reply to be addressed to all three of you.
Answer me honestly, when reading a fairy tale, do you really stop to consider what’s written there, qualify its worth as evidence, and compare it to everything else you know that might contradict it, before making the decision that the probability of the fairy tale being true is extremely low? Do you really not just dismiss it offhand as not true without a second thought?
When I pick up a work of fiction, I do not spend time assessing its veracity. If I read a book of equally fantastic claims which purports to be true, I do spend a little time. You might want to peruse bounded rationality for an overview.
So you would then agree that merely the fact that something is written SOMEWHERE, does not automatically qualify it as evidence?
(Incidentally that is my original point, which in spite of seeming as common sense as common sense can be, has attracted a surprising amount of disagreement.)
So you would then agree that merely the fact that something is written SOMEWHERE, does not automatically qualify it as evidence?
You have to specify what it purports to be evidence of before I can give you an answer that isn’t a tangent.
Edited to add: Maybe I can do better than the above sentence. I affirm that the existence of this book is negligible but not strictly zero evidence for the claims detailed therein.
At this point I’m not sure what we can do other than agree to disagree. I do not consider a random article from an obscure source on the internet to be evidence of anything.
There may be sense in which this is common sense, but you were purposely using it tendentiously, which is why people responded in the technical way that they did.
Eliezer said that he read something “somewhere”, obviously intending to say that he read it somewhere that he considered trustworthy at the time, not in a fairy tale.
Well, what can I say? I simply don’t consider the vague recollection of reading something somewhere credible evidence of anything, and I stand by that. However, the amount of people that took issue with this statement did open my eyes to the fact that the definition of word “evidence” is not as clear cut as I thought it to be. Not sure if there’s any way to resolve this difference of opinion though.
As noted by jimrandomh, saying ‘credible evidence’ does make an effort to differentiate between different sorts of evidence. If your claim was simply that reading something was not evidence, then you should not have to qualify the word when you use it now. I imagine for those of us who seem to be disagreeing with you, we would agree that that does not constitute ‘credible evidence’ for some values of ‘credible’.
That’s really clever. I always thought that “credible evidence” was a bit redundant actually. I just used as a figure of speech without thinking about, but according to my definition of evidence that it has to be credible is pretty much implicit. It has been made abundantly clear to me, however, that this community’s definition differs substantially, so that’s the definition I will use when posting here going forward.
The easy solution is to stop arguing about the definition of evidence. This community uses it to mean one thing, you’re using it to mean something else, and any sort of conflict goes away as soon as people make clear which definition they’re using. Since this community already has an accepted definition, you would be safe in assuming that that definition is what other posters here have in mind when they use the word “evidence”. By the same token, you should probably find a more precise way to refer to the definition of evidence that you are using in order to avoid being misinterpreted.
Sticking an adjective in front of the word evidence seems to work. “Evidence” includes things that give you 10^-15 bits of information; on the other hand “good evidence”, “usable evidence” and “credible evidence” all imply that the strength of the evidence is at least not exponentially tiny.
I thought that “evidence”, unmodified, would mean non-trivial evidence; otherwise, everything has to count as evidence because it will have some connection to the hypothesis, however weak. To specify a kind of evidence that includes the 1e-15 bit case, I think you would need to say “weak evidence” or “very weak evidence”.
But I’m not the authority on this: How do others here interpret the term “evidence” (in the Bayesian sense) when it’s unmodified?
I would if I were talking to a Bayesian, interpret it as meaning something where a “B is evidence for A” if rough calculation shows that P(A|B) > P(A). I don’t generally expect rationalists to even mention individual data points unless P(A|B)/P(A) is large, but if someone else gave the data as an example, then I wouldn’t expect it to be necessarily large if a Bayesian referred to as evidence. So for example, I could see a Bayesian asserting that the writing of the Bible is evidence for a global flood some 5000 years ago, but I’d be deeply surprised if a Bayesian brought this up in almost any context because the evidence is so weak (in this case P(A|B)>P(A) but P(A|B)/P(A) is very close to 1).
I agree, this sounds exactly right to me. Unfortunately, I remember that in a lot of Robin Hanson’s earlier OvercomingBias posts, my reaction to them would be, “Yes, B is technically evidence in favor of A, but it’s extremely weak—why even mention it?” For example, Suicide Rock.
(I think I have a picture of one of those somewhere...)
I have encountered a number of disputes that revolve around using these two different senses of the word, and am nonetheless blindsided by them consistently.
I try to always specify the strength of evidence in some sense when using the word. I think when I do use it unmodified I tend to use it in the technical sense (including even weak evidence).
It would be odd if ‘evidence’ excluded weak evidence, since then ‘weak evidence’ would be a contradiction in terms, or you could see people arguing things like “When I said ‘weak evidence’ I didn’t mean the 1e-15 bit case, since that’s not evidence at all!”
That’s fair enough. However, judging by what I’ve read, this community’s definition of evidence seems to constitute just about anything ever written about anything. How would you then differentiate evidence, from rumor, hearsay, speculation, etc.?
Well that clarifies things quite a bit. I find this definition of evidence surprising, especially in this community, but very interesting. I’ll have to sleep on it. Thank you for the references.
Rumor, hearsay, etc. falls under our definition of evidence, just weak evidence, or probably very indirect (for example, if there is a rumor that A, it might constitute evidence against A being true, given other things you know).
Immediate observation is only that something is written. That it’s also true is a theoretical hypothesis about that immediate observation. That what you are reading is a fairy tale is evidence against the things written there being true, so the theory that what’s written in a fairy tale is true is weak. On the other hand, the fact that you observe the words of a given fairy tale is strong evidence that the person (author) whose name is printed on the cover really existed.
All that is indisputably true. But you didn’t really answer my question on whether or not you give enough consideration to what’s written in a fairy tale (not whether or not it’s written, not who it’s written by, but the actual claims made therein) to truly consider it evidence to be incorporated into or excluded from your model of the world.
But you didn’t really answer my question on whether or not you give enough consideration to what’s written in a fairy tale (not whether or not it’s written, not who it’s written by, but the actual claims made therein) to truly consider it evidence to be incorporated into or excluded from your model of the world.
That is because it is a bad question and one of a form for which you have already received responses.
Evidence isn’t usually something you “include” in your model of the world, it’s something you use to categorize models of the world into correct and incorrect ones. Evidence is usually something not interesting in itself, but interesting instrumentally because of the things it’s connected to (caused by).
Vladimir, Cyan, and jimrandomh, since you essentially said the same thing, consider this reply to be addressed to all three of you.
Answer me honestly, when reading a fairy tale, do you really stop to consider what’s written there, qualify its worth as evidence, and compare it to everything else you know that might contradict it, before making the decision that the probability of the fairy tale being true is extremely low? Do you really not just dismiss it offhand as not true without a second thought?
No, but only because that would be cognitively burdensome. We’re boundedly rational.
When I pick up a work of fiction, I do not spend time assessing its veracity. If I read a book of equally fantastic claims which purports to be true, I do spend a little time. You might want to peruse bounded rationality for an overview.
So you would then agree that merely the fact that something is written SOMEWHERE, does not automatically qualify it as evidence?
(Incidentally that is my original point, which in spite of seeming as common sense as common sense can be, has attracted a surprising amount of disagreement.)
You have to specify what it purports to be evidence of before I can give you an answer that isn’t a tangent.
Edited to add: Maybe I can do better than the above sentence. I affirm that the existence of this book is negligible but not strictly zero evidence for the claims detailed therein.
At this point I’m not sure what we can do other than agree to disagree. I do not consider a random article from an obscure source on the internet to be evidence of anything.
There may be sense in which this is common sense, but you were purposely using it tendentiously, which is why people responded in the technical way that they did.
Eliezer said that he read something “somewhere”, obviously intending to say that he read it somewhere that he considered trustworthy at the time, not in a fairy tale.
Well, what can I say? I simply don’t consider the vague recollection of reading something somewhere credible evidence of anything, and I stand by that. However, the amount of people that took issue with this statement did open my eyes to the fact that the definition of word “evidence” is not as clear cut as I thought it to be. Not sure if there’s any way to resolve this difference of opinion though.
As noted by jimrandomh, saying ‘credible evidence’ does make an effort to differentiate between different sorts of evidence. If your claim was simply that reading something was not evidence, then you should not have to qualify the word when you use it now. I imagine for those of us who seem to be disagreeing with you, we would agree that that does not constitute ‘credible evidence’ for some values of ‘credible’.
That’s really clever. I always thought that “credible evidence” was a bit redundant actually. I just used as a figure of speech without thinking about, but according to my definition of evidence that it has to be credible is pretty much implicit. It has been made abundantly clear to me, however, that this community’s definition differs substantially, so that’s the definition I will use when posting here going forward.
The easy solution is to stop arguing about the definition of evidence. This community uses it to mean one thing, you’re using it to mean something else, and any sort of conflict goes away as soon as people make clear which definition they’re using. Since this community already has an accepted definition, you would be safe in assuming that that definition is what other posters here have in mind when they use the word “evidence”. By the same token, you should probably find a more precise way to refer to the definition of evidence that you are using in order to avoid being misinterpreted.
Sticking an adjective in front of the word evidence seems to work. “Evidence” includes things that give you 10^-15 bits of information; on the other hand “good evidence”, “usable evidence” and “credible evidence” all imply that the strength of the evidence is at least not exponentially tiny.
I thought that “evidence”, unmodified, would mean non-trivial evidence; otherwise, everything has to count as evidence because it will have some connection to the hypothesis, however weak. To specify a kind of evidence that includes the 1e-15 bit case, I think you would need to say “weak evidence” or “very weak evidence”.
But I’m not the authority on this: How do others here interpret the term “evidence” (in the Bayesian sense) when it’s unmodified?
I would if I were talking to a Bayesian, interpret it as meaning something where a “B is evidence for A” if rough calculation shows that P(A|B) > P(A). I don’t generally expect rationalists to even mention individual data points unless P(A|B)/P(A) is large, but if someone else gave the data as an example, then I wouldn’t expect it to be necessarily large if a Bayesian referred to as evidence. So for example, I could see a Bayesian asserting that the writing of the Bible is evidence for a global flood some 5000 years ago, but I’d be deeply surprised if a Bayesian brought this up in almost any context because the evidence is so weak (in this case P(A|B)>P(A) but P(A|B)/P(A) is very close to 1).
I agree, this sounds exactly right to me. Unfortunately, I remember that in a lot of Robin Hanson’s earlier OvercomingBias posts, my reaction to them would be, “Yes, B is technically evidence in favor of A, but it’s extremely weak—why even mention it?” For example, Suicide Rock.
(I think I have a picture of one of those somewhere...)
Hmm. Maybe the strength of the evidence isn’t the right thing to use, but rather the confidence with which we know the sign of the correlation.
I’m sympathetic to both views.
I have encountered a number of disputes that revolve around using these two different senses of the word, and am nonetheless blindsided by them consistently.
I try to always specify the strength of evidence in some sense when using the word. I think when I do use it unmodified I tend to use it in the technical sense (including even weak evidence).
It would be odd if ‘evidence’ excluded weak evidence, since then ‘weak evidence’ would be a contradiction in terms, or you could see people arguing things like “When I said ‘weak evidence’ I didn’t mean the 1e-15 bit case, since that’s not evidence at all!”
That’s fair enough. However, judging by what I’ve read, this community’s definition of evidence seems to constitute just about anything ever written about anything. How would you then differentiate evidence, from rumor, hearsay, speculation, etc.?
The wiki should be a good starting point for answering this question. What is Evidence? may also be helpful.
Short version: rumor, hearsay, and speculation are evidence, albeit of a very weak variety.
Well that clarifies things quite a bit. I find this definition of evidence surprising, especially in this community, but very interesting. I’ll have to sleep on it. Thank you for the references.
Rumor, hearsay, etc. falls under our definition of evidence, just weak evidence, or probably very indirect (for example, if there is a rumor that A, it might constitute evidence against A being true, given other things you know).
Immediate observation is only that something is written. That it’s also true is a theoretical hypothesis about that immediate observation. That what you are reading is a fairy tale is evidence against the things written there being true, so the theory that what’s written in a fairy tale is true is weak. On the other hand, the fact that you observe the words of a given fairy tale is strong evidence that the person (author) whose name is printed on the cover really existed.
All that is indisputably true. But you didn’t really answer my question on whether or not you give enough consideration to what’s written in a fairy tale (not whether or not it’s written, not who it’s written by, but the actual claims made therein) to truly consider it evidence to be incorporated into or excluded from your model of the world.
That is because it is a bad question and one of a form for which you have already received responses.
Evidence isn’t usually something you “include” in your model of the world, it’s something you use to categorize models of the world into correct and incorrect ones. Evidence is usually something not interesting in itself, but interesting instrumentally because of the things it’s connected to (caused by).