Second example. “Simon Baron-Cohen playing Motte-Bailey with the “extreme male brain” theory of autism.” Let’s see… It seems uncontroversial (among the participants in this discussion) that there are dimensions on which male and female brains differ (on average), and on which autists are (on average) skewed towards the male side, and that this includes the empathizing and systematizing dimensions.
Quick update!
I found that OpenPsychometrics has a dataset for the EQ/SQ tests. Unfortunately, there seems to be a problem for the data with the EQ items, but I just ran a factor analysis for the SQ items to take a closer look at your claims here.
There appeared to be 3 or 4 factors underlying the correlations on the SQ test, which I’d roughly call “Technical interests”, “Nature interests”, “Social difficulties” and “Jockyness”. I grabbed the top loading items for each of the factors, and got this correlation matrix:
The correlations between the technical interests and nature interests plausibly reflects the notion that Systematizing is a thing, though I suspect that it could also be found to correlate with all sorts of other things that would not be considered Systematizing? Like non-Systematizing ways of interacting with nature. Idk though.
The sex differences in the items was limited to the technical interests, rather than than also covering the nature interests. This does not fit a simple model of a sex difference in general Systematizing, but it does fit a model where the items are biased towards men but there is not much sex difference in general Systematizing.
I would be inclined to think that the Social difficulties items correlate negatively with Empathizing Quotient or positively with Autism Spectrum Quotient. If we are interested in the correlations between general Systematizing and these other factors, then this could bias the comparisons. On the other hand, the Social difficulties items were not very strongly correlated with the overall SQ score, so maybe not.
I can’t immediately think of any comments for the Jockyness items.
Overall, I strongly respect the fact that he made many of the items very concrete, but I now also feel like I have proven that the gender differences on Systematizing to be driven by psychometric shenanigans, and I strongly expect to find that many of the other associations are also driven by psychometric shenanigans.
I’ve sent an email asking OpenPsychometrics to export the Empathizing Quotient items too. If he does so, I hope to write a top-level post explaining my issues with the psychometrics here.
Hm, actually I semi-retract this; the OpenPsychometrics data seems to be based on the original Systematizing Quotient, whereas there seems to be a newer one called Systematizing Quotient-Revised, which is supposedly more gender-neutral. Not sure where I can get data on this, though. Will go looking.
Edit: Like I am still pretty suspicious about the SQ-R. I just don’t have explicit proof that it is flawed.
Oops, upon reading more about the SQ, I should correct myself:
Some of the items, such as S16, are “filler items” which are not counted as part of the score; these are disproportionately part of the “Social difficulties” and “Jockyness” factors, so that probably reduces the amount of bias that can be introduced by those items, and it also also explains why they don’t correlate very much with the overall SQ scores.
But some of the items for these factors, such as S31, are not filler items, and instead get counted for the test, presumably because they have cross-loadings on the Systematizing factor. So the induced bias is probably not zero.
If I get the data from OpenPsychometrics, I will investigate in more detail.
Since I don’t have data on the EQ, here’s a study where someone else worked with it. They found that the EQ had three factors, which they named “Cognitive Empathy”, “Emotional Empathy” and “Social Skills”. The male-female difference was driven by “Emotional Empathy” (d=1), whereas the autistic-allistic difference was driven by “Social Skills” (d=1.3). The converse differences were much smaller, 0.24 and 0.66. As such, it seems likely that the EQ lumps together two different kinds of “empathizing”, one of which is feminine and one of which is allistic.
Quick update!
I found that OpenPsychometrics has a dataset for the EQ/SQ tests. Unfortunately, there seems to be a problem for the data with the EQ items, but I just ran a factor analysis for the SQ items to take a closer look at your claims here.
There appeared to be 3 or 4 factors underlying the correlations on the SQ test, which I’d roughly call “Technical interests”, “Nature interests”, “Social difficulties” and “Jockyness”. I grabbed the top loading items for each of the factors, and got this correlation matrix:
The correlations between the technical interests and nature interests plausibly reflects the notion that Systematizing is a thing, though I suspect that it could also be found to correlate with all sorts of other things that would not be considered Systematizing? Like non-Systematizing ways of interacting with nature. Idk though.
The sex differences in the items was limited to the technical interests, rather than than also covering the nature interests. This does not fit a simple model of a sex difference in general Systematizing, but it does fit a model where the items are biased towards men but there is not much sex difference in general Systematizing.
I would be inclined to think that the Social difficulties items correlate negatively with Empathizing Quotient or positively with Autism Spectrum Quotient. If we are interested in the correlations between general Systematizing and these other factors, then this could bias the comparisons. On the other hand, the Social difficulties items were not very strongly correlated with the overall SQ score, so maybe not.
I can’t immediately think of any comments for the Jockyness items.
Overall, I strongly respect the fact that he made many of the items very concrete, but I now also feel like I have proven that the gender differences on Systematizing to be driven by psychometric shenanigans, and I strongly expect to find that many of the other associations are also driven by psychometric shenanigans.
I’ve sent an email asking OpenPsychometrics to export the Empathizing Quotient items too. If he does so, I hope to write a top-level post explaining my issues with the psychometrics here.
Hm, actually I semi-retract this; the OpenPsychometrics data seems to be based on the original Systematizing Quotient, whereas there seems to be a newer one called Systematizing Quotient-Revised, which is supposedly more gender-neutral. Not sure where I can get data on this, though. Will go looking.
Edit: Like I am still pretty suspicious about the SQ-R. I just don’t have explicit proof that it is flawed.
Am I gonna have to collect the data myself? I might have to collect the data myself...
Oops, upon reading more about the SQ, I should correct myself:
Some of the items, such as S16, are “filler items” which are not counted as part of the score; these are disproportionately part of the “Social difficulties” and “Jockyness” factors, so that probably reduces the amount of bias that can be introduced by those items, and it also also explains why they don’t correlate very much with the overall SQ scores.
But some of the items for these factors, such as S31, are not filler items, and instead get counted for the test, presumably because they have cross-loadings on the Systematizing factor. So the induced bias is probably not zero.
If I get the data from OpenPsychometrics, I will investigate in more detail.
Since I don’t have data on the EQ, here’s a study where someone else worked with it. They found that the EQ had three factors, which they named “Cognitive Empathy”, “Emotional Empathy” and “Social Skills”. The male-female difference was driven by “Emotional Empathy” (d=1), whereas the autistic-allistic difference was driven by “Social Skills” (d=1.3). The converse differences were much smaller, 0.24 and 0.66. As such, it seems likely that the EQ lumps together two different kinds of “empathizing”, one of which is feminine and one of which is allistic.