I agree that the counterarguments to AI risk seem quite poor (to me), and this is some evidence in favor of AI risk. However, since humans are biased towards not being convinced by any counterargument to something they strongly believe, this limits how much we can use the unconvincingness of counterarguments to AI risk to update in favor of AI risk.
“The general idea is that it’s absolutely threatening to admit you’re wrong,” says political scientist Brendan Nyhan, the lead researcher on the Michigan study. The phenomenon — known as “backfire” — is “a natural defense mechanism to avoid that cognitive dissonance.”
(I’m sure this was linked on LW before, but couldn’t find it quickly …)
So the proposed “failure of counter-arguments argument” falls afoul of errors in human thinking.
The backfire effect, as far as I know, has never been replicated and bears all the hallmarks of being your classic counterintuitive psych finding which will disappear the moment anyone looks too hard at it. I’ve been trying to discourage people from citing it...
Stereotype threat looks extremely questionable to me, and dual n-back is more or less finished as far as I’m concerned. Those are the only two famous findings which I can think of off-hand; did you have any specific ones in mind?
I feel obligated to point out that the backfire effect, even in the original paper, applied only to a few zealots, and not everyone and not relatively moderate subjects. So your obvious joke is not itself consistent with the paper.
I agree that the counterarguments to AI risk seem quite poor (to me), and this is some evidence in favor of AI risk. However, since humans are biased towards not being convinced by any counterargument to something they strongly believe, this limits how much we can use the unconvincingness of counterarguments to AI risk to update in favor of AI risk.
Indeed. It has been experimentally tested that strong counterarguments are less likely to convince a strong believer.
(I’m sure this was linked on LW before, but couldn’t find it quickly …)
So the proposed “failure of counter-arguments argument” falls afoul of errors in human thinking.
The backfire effect, as far as I know, has never been replicated and bears all the hallmarks of being your classic counterintuitive psych finding which will disappear the moment anyone looks too hard at it. I’ve been trying to discourage people from citing it...
Nononono, don’t do that! It’ll just make them believe it more strongly.
(The nice thing is that actually your attempts at discouragement will, roughly speaking, work if and only if the discouragement is correct!)
What other (popular) psychology findings do you think won’t hold up?
Stereotype threat looks extremely questionable to me, and dual n-back is more or less finished as far as I’m concerned. Those are the only two famous findings which I can think of off-hand; did you have any specific ones in mind?
Or maybe the evidence for the effect is so strong that you refuse to believe it. :)
I feel obligated to point out that the backfire effect, even in the original paper, applied only to a few zealots, and not everyone and not relatively moderate subjects. So your obvious joke is not itself consistent with the paper.