Thanks for the thoughtful comment. I modified my post to address it.
If there is one single strong piece of evidence, then we should update on that single strong piece of evidence, neither being happy nor unhappy about its singular nature.
The question is: given a single strong piece of evidence in favor of A and several weak pieces of evidence against A, how should one weigh the evidence on both sides? Obviously this will depend on the particulars.
Broadly, my earlier approach was to focus on the single strong piece of evidence in favor of A, and ignore the weak pieces of evidence against A, whereas now my approach is to give some weight to the several weak pieces evidence, and to allow them it to overwhelm the single strong piece of evidence in favor of A in some situations.
I’m not sure if your post is mostly concerned with argument presentation, since you state that “the ‘single relatively strong argument’ approach to assessing the above claim is to try to synthesize as many of the above weak arguments and counterarguments as possible, into a single relatively strong argument.”—That’s not so much one strong argument, as simply taking all the weak arguments then presenting them in an intermixed and baked-together fashion. I still see the component arguments, it’s just that their presentation has become less clean.
This reflects poor presentation on my part. When I said “synthesize,” the sort of thing that I had in mind is to say “Correlation is not causation. Conventional wisdom is probably rooted in mistaking correlation for causation. Therefore we should ignore conventional wisdom.” This implicitly throws out the possibility that conventional wisdom isn’t rooted in mistaking correlation for causation.
Doesn’t “one strong argument” mean something other “many weak arguments intermixed”?
In focusing on a single strong argument to the exclusion of other arguments, one is implicitly rejecting the weak arguments, and so doing so constitutes an implicit attempt to synthesize the evidence.
Broadly, my earlier approach was to focus on the single strong piece of evidence in favor of A, and ignore the weak pieces of evidence against A, whereas now my approach is to give some weight to the several weak pieces evidence, and to allow them it to overwhelm the single strong piece of evidence in favor of A in some situations.
That seems very commonsensical and should be very uncontroversial.
Well, it’s hard to be controversial when saying “you should give some weight even to weak evidence”. As opposed to evidence in favor of a position which would not shift your prior at all? That would be a contradictio in terminis, since evidence in favor is defined as shifting your prior to some degree). “In favor, even weakly” = P(A|evidence) > P(A).
What about the second part, which—because of the “in some situations” qualifier—is equivalent to “there exist situations such that a number of small belief shifts can overcome one large belief shift. Common sense as well, since it just restates that adding enough (non-diminishing and ergo terms of a diverging series) epsilons will overcome any fixed number, however large.
As a general rule, we should not expect there to be strong pieces of evidence in favor of a false position. Not strong in the sense of “well presented”, or “given by an authority figure”, but strong as evaluated by a person knowledgeable in the field. In physics, you won’t get to 5 sigma using many small updates, you’ll have a couple of strong results. There may be the occasional strange circumstance in which there appears to be actual strong evidence against a position which (the position) later turns out to be correct. However, there always will be a lot of weak evidence (at various values of ‘weak’) for or against anything, it’s just too easy to come up with using motivated cognition.
So we should take note if there’s strong evidence involved in any issue, but with “you should not outright ignore weak evidence”, we can all be friends.
That seems very commonsensical and should be very uncontroversial.
I knew this in the abstract, but wasn’t adhering to it properly in practice. See my remarks about the shift in my beliefs about Penrose’s views on consciousness.
As a general rule, we should not expect there to be strong pieces of evidence in favor of a false position.
But there are often apparently strong pieces of evidence in favor of a false position. That’s the point of the latter half of the “Major weaknesses of the “single relatively strong argument” approach section of my post.
In practice, it’s often the case that all we have is weak evidence — the situation is just that some evidence is weaker than other evidence. It can be easy to deceive oneself into thinking that the relatively strong evidence is stronger than it is.
However, there always will be a lot of weak evidence (at various values of ‘weak’) for or against anything, it’s just too easy to come up with using motivated cognition.
I agree if you’re talking about “sufficiently weak” evidence. But consider the example of quantitative careers and earnings in my post. I believe that the individual arguments supporting it are genuinely weak, but that there are fewer arguments of the same strength against the claim, so that there’s not much of a risk of motivated cognition skewing the conclusion. Do you disagree?
Thanks for the thoughtful comment. I modified my post to address it.
The question is: given a single strong piece of evidence in favor of A and several weak pieces of evidence against A, how should one weigh the evidence on both sides? Obviously this will depend on the particulars.
Broadly, my earlier approach was to focus on the single strong piece of evidence in favor of A, and ignore the weak pieces of evidence against A, whereas now my approach is to give some weight to the several weak pieces evidence, and to allow them it to overwhelm the single strong piece of evidence in favor of A in some situations.
This reflects poor presentation on my part. When I said “synthesize,” the sort of thing that I had in mind is to say “Correlation is not causation. Conventional wisdom is probably rooted in mistaking correlation for causation. Therefore we should ignore conventional wisdom.” This implicitly throws out the possibility that conventional wisdom isn’t rooted in mistaking correlation for causation.
In focusing on a single strong argument to the exclusion of other arguments, one is implicitly rejecting the weak arguments, and so doing so constitutes an implicit attempt to synthesize the evidence.
That seems very commonsensical and should be very uncontroversial.
Well, it’s hard to be controversial when saying “you should give some weight even to weak evidence”. As opposed to evidence in favor of a position which would not shift your prior at all? That would be a contradictio in terminis, since evidence in favor is defined as shifting your prior to some degree). “In favor, even weakly” = P(A|evidence) > P(A).
What about the second part, which—because of the “in some situations” qualifier—is equivalent to “there exist situations such that a number of small belief shifts can overcome one large belief shift. Common sense as well, since it just restates that adding enough (non-diminishing and ergo terms of a diverging series) epsilons will overcome any fixed number, however large.
As a general rule, we should not expect there to be strong pieces of evidence in favor of a false position. Not strong in the sense of “well presented”, or “given by an authority figure”, but strong as evaluated by a person knowledgeable in the field. In physics, you won’t get to 5 sigma using many small updates, you’ll have a couple of strong results. There may be the occasional strange circumstance in which there appears to be actual strong evidence against a position which (the position) later turns out to be correct. However, there always will be a lot of weak evidence (at various values of ‘weak’) for or against anything, it’s just too easy to come up with using motivated cognition.
So we should take note if there’s strong evidence involved in any issue, but with “you should not outright ignore weak evidence”, we can all be friends.
I knew this in the abstract, but wasn’t adhering to it properly in practice. See my remarks about the shift in my beliefs about Penrose’s views on consciousness.
But there are often apparently strong pieces of evidence in favor of a false position. That’s the point of the latter half of the “Major weaknesses of the “single relatively strong argument” approach section of my post.
In practice, it’s often the case that all we have is weak evidence — the situation is just that some evidence is weaker than other evidence. It can be easy to deceive oneself into thinking that the relatively strong evidence is stronger than it is.
I agree if you’re talking about “sufficiently weak” evidence. But consider the example of quantitative careers and earnings in my post. I believe that the individual arguments supporting it are genuinely weak, but that there are fewer arguments of the same strength against the claim, so that there’s not much of a risk of motivated cognition skewing the conclusion. Do you disagree?