Robin Hanson also posted on this study—his post contains a longer passage from the paper.
The New Yorker article is misleading—it makes it sound like smarter people are more biased, but that is not what the study found on the various particular biases that it measured, only on the meta-bias of the bias blind spot (and even that is iffy—more on that below). Previous research has generally found that smarter people are either less biased or equally biased, depending on the bias; I don’t know of any cases where smarter people are more biased.
Did smarter people show a stronger bias blind spot in this study? The authors of the paper say (in the abstract) that the bias blind spot was not attenuated by higher cognitive ability, and “If anything, a larger bias blind spot was associated with higher cognitive ability.” Presumably, “if anything” means that the association was weak and not consistently statistically significant across their various measures. Otherwise they’d just say that the bias blind spot was bigger, rather than not smaller, and if anything bigger. Similarly, in the passage that Robin quotes, they only highlight the novelty of their result by saying “the bias blind spot joins a small group of other effects such as myside bias and noncausal base-rate neglect in being unmitigated by increases in intelligence.” I haven’t read the paper yet (I can’t find it online), it sounds like it only provides weak evidence for a positive correlation between cognitive ability and the bias blind spot.
After reading the paper (thanks for the link, VincentYu), it turns out that the evidence was stronger than I thought. The authors were being conservative to use the “if anything” language—I think more conservative than they needed to be. In study 1, their composite measure of the bias blind spot had a statistically significant positive correlation with each of their 4 intelligence-related measures (SAT, CRT, need for cognition, actively open-minded thinking), with correlations ranging from .096 (p<.05) to .260 (p<.001). It’s a relatively weak correlation, but still pretty strong evidence that it’s a real correlation rather than noise. For 4 of the 7 particular cognitive biases that they studied, the bias blind spot had a statistically significant correlation with SAT scores (see Jack’s comment for more). In study 2, the correlation between the CRT and the bias blind spot composite was almost identical (r=.102, compared with .096 in study 1), but not quite statistically significant (p=.10) because of a smaller sample size. So I think it would be fair to state the claim more strongly than they do—smarter people do tend to think that they’re less biased (at least within this study).
However, I don’t agree with calling this a “larger bias blind spot.” The bias blind spot is an instance of the better-than-average effect. It was measured by describing a specific bias to university students, asking them how susceptible to that bias they think that they are, and asking them how susceptible to that bias they think that other students at their university. People tend to say that they are less susceptible to the bias than other people are. This cannot be true on average (since people must be average on average), so it is a bias.
But studying individual differences in the bias blind spot is tricky because any particular individual who claims to be less biased than average may actually be less biased (rather than being more metabiased about how biased they are). I’m not sure if any other research on the bias blind spot has looked at individual differences; the main focus has been to look at differences between biases (which biases do people freely admit to having, and which do they have the largest blind spot for?). The bias blind spot measures don’t have any clear standard for accuracy, other than the fact that self and other must be equally biased on average as long as they’re sampled from the same population.
When you’re comparing people who have high SAT scores with people who have lower SAT scores (as this study did), you don’t have that because you’re drawing from two different populations, and people with higher SAT scores do tend to be less biased (on many particular cognitive biases, though not all of them). If you had a really good, thorough measure of how susceptible to cognitive biases people are (one which covered a wide range of different biases, and which included lots of questions so that it could reliably identify individual differences among the noise), then you would presumably find that smarter people tend to score as less biased. So it’s no surprise that smarter people tend to think that they’re less biased—that seems to reflect accurate self-knowledge (and maybe also accurate theories, e.g. “This seems like the kind of thing that smart people would be better at”). It may be the case that smart people have trouble distinguishing the biases that their intelligence helps them avoid from the biases where their intelligence doesn’t help, but that’s not the same thing as having a larger bias blind spot (i.e., a stronger better-than-average effect with respect to how biased you are).
This study didn’t include a thorough test of how biased people were—it picked a few specific biases and had 1 question each testing how susceptible people were to that bias (2 for anchoring). And this study actually did find that smarter people were less biased—on 3 of the 9 questions (across the 2 studies) there was a statistically significant relationship between intelligence and the bias (those with higher SAT scores had less outcome bias, less myside bias, and less anchoring bias on one of the two anchoring questions). The authors downplayed those effects because they only accounted for a tiny percentage of the variance, but on a noisy (single-item) measure I’d only expect a variable like this to explain a small amount of variance (and the correlation between intelligence and the “bias blind spot” composite also had a small effect size). Interestingly, outcome bias and myside bias were two of the biases where SAT did not have a statistically significant relationship with their bias blind spot measure—so smarter people did seem to be less biased on the whole, and they thought that they would be less biased, but they didn’t know which biases they would be less susceptible to.
Out of 28 correlations 24 were positive and 17 were statistically significant, six of the 7 biases tested showed the relationship (the one that didn’t isn’t really a cognitive bias). For the Bias Blind spot composite score SATs were significant at p<.001, N=482. The authors are conservative in their assessment of the results. To provide some context West and Stanovich have spent the last decade examining correlations between intelligence and cognitive biases.
Upvoted. Based on reading the abstract and Lehrer’s article, I’d expect only a 50% chance or so that these results will replicate, and even less so that they will generalize to other biases.
Which results? The results of the paper, which found that cognitive sophistication did not attenuate the bias blind spot or the slight positive correlation between cognitive sophistication and bias blind spot that the journalist decided to focus his article on?
Robin Hanson also posted on this study—his post contains a longer passage from the paper.
The New Yorker article is misleading—it makes it sound like smarter people are more biased, but that is not what the study found on the various particular biases that it measured, only on the meta-bias of the bias blind spot (and even that is iffy—more on that below). Previous research has generally found that smarter people are either less biased or equally biased, depending on the bias; I don’t know of any cases where smarter people are more biased.
Did smarter people show a stronger bias blind spot in this study? The authors of the paper say (in the abstract) that the bias blind spot was not attenuated by higher cognitive ability, and “If anything, a larger bias blind spot was associated with higher cognitive ability.” Presumably, “if anything” means that the association was weak and not consistently statistically significant across their various measures. Otherwise they’d just say that the bias blind spot was bigger, rather than not smaller, and if anything bigger. Similarly, in the passage that Robin quotes, they only highlight the novelty of their result by saying “the bias blind spot joins a small group of other effects such as myside bias and noncausal base-rate neglect in being unmitigated by increases in intelligence.” I haven’t read the paper yet (I can’t find it online), it sounds like it only provides weak evidence for a positive correlation between cognitive ability and the bias blind spot.
After reading the paper (thanks for the link, VincentYu), it turns out that the evidence was stronger than I thought. The authors were being conservative to use the “if anything” language—I think more conservative than they needed to be. In study 1, their composite measure of the bias blind spot had a statistically significant positive correlation with each of their 4 intelligence-related measures (SAT, CRT, need for cognition, actively open-minded thinking), with correlations ranging from .096 (p<.05) to .260 (p<.001). It’s a relatively weak correlation, but still pretty strong evidence that it’s a real correlation rather than noise. For 4 of the 7 particular cognitive biases that they studied, the bias blind spot had a statistically significant correlation with SAT scores (see Jack’s comment for more). In study 2, the correlation between the CRT and the bias blind spot composite was almost identical (r=.102, compared with .096 in study 1), but not quite statistically significant (p=.10) because of a smaller sample size. So I think it would be fair to state the claim more strongly than they do—smarter people do tend to think that they’re less biased (at least within this study).
However, I don’t agree with calling this a “larger bias blind spot.” The bias blind spot is an instance of the better-than-average effect. It was measured by describing a specific bias to university students, asking them how susceptible to that bias they think that they are, and asking them how susceptible to that bias they think that other students at their university. People tend to say that they are less susceptible to the bias than other people are. This cannot be true on average (since people must be average on average), so it is a bias.
But studying individual differences in the bias blind spot is tricky because any particular individual who claims to be less biased than average may actually be less biased (rather than being more metabiased about how biased they are). I’m not sure if any other research on the bias blind spot has looked at individual differences; the main focus has been to look at differences between biases (which biases do people freely admit to having, and which do they have the largest blind spot for?). The bias blind spot measures don’t have any clear standard for accuracy, other than the fact that self and other must be equally biased on average as long as they’re sampled from the same population.
When you’re comparing people who have high SAT scores with people who have lower SAT scores (as this study did), you don’t have that because you’re drawing from two different populations, and people with higher SAT scores do tend to be less biased (on many particular cognitive biases, though not all of them). If you had a really good, thorough measure of how susceptible to cognitive biases people are (one which covered a wide range of different biases, and which included lots of questions so that it could reliably identify individual differences among the noise), then you would presumably find that smarter people tend to score as less biased. So it’s no surprise that smarter people tend to think that they’re less biased—that seems to reflect accurate self-knowledge (and maybe also accurate theories, e.g. “This seems like the kind of thing that smart people would be better at”). It may be the case that smart people have trouble distinguishing the biases that their intelligence helps them avoid from the biases where their intelligence doesn’t help, but that’s not the same thing as having a larger bias blind spot (i.e., a stronger better-than-average effect with respect to how biased you are).
This study didn’t include a thorough test of how biased people were—it picked a few specific biases and had 1 question each testing how susceptible people were to that bias (2 for anchoring). And this study actually did find that smarter people were less biased—on 3 of the 9 questions (across the 2 studies) there was a statistically significant relationship between intelligence and the bias (those with higher SAT scores had less outcome bias, less myside bias, and less anchoring bias on one of the two anchoring questions). The authors downplayed those effects because they only accounted for a tiny percentage of the variance, but on a noisy (single-item) measure I’d only expect a variable like this to explain a small amount of variance (and the correlation between intelligence and the “bias blind spot” composite also had a small effect size). Interestingly, outcome bias and myside bias were two of the biases where SAT did not have a statistically significant relationship with their bias blind spot measure—so smarter people did seem to be less biased on the whole, and they thought that they would be less biased, but they didn’t know which biases they would be less susceptible to.
Out of 28 correlations 24 were positive and 17 were statistically significant, six of the 7 biases tested showed the relationship (the one that didn’t isn’t really a cognitive bias). For the Bias Blind spot composite score SATs were significant at p<.001, N=482. The authors are conservative in their assessment of the results. To provide some context West and Stanovich have spent the last decade examining correlations between intelligence and cognitive biases.
Upvoted. Based on reading the abstract and Lehrer’s article, I’d expect only a 50% chance or so that these results will replicate, and even less so that they will generalize to other biases.
Which results? The results of the paper, which found that cognitive sophistication did not attenuate the bias blind spot or the slight positive correlation between cognitive sophistication and bias blind spot that the journalist decided to focus his article on?