The IQ numbers have time and time again answered every challenge raised against them and should be presumed accurate.
What if the people who have taken IQ tests are on average smarter than the people who haven’t? My impression is that people mostly take IQ tests when they’re somewhat extreme: either low and trying to qualify for assistive services or high and trying to get “gifted” treatment. If we figure lesswrong draws mostly from the high end, then we should expect the IQ among test-takers to be higher than what we would get if we tested random people who had not previously been tested.
The IQ Question read: “Please give the score you got on your most recent PROFESSIONAL, SCIENTIFIC IQ test—no Internet tests, please! All tests should have the standard average of 100 and stdev of 15.”
Among the subset of people making their data public (n=1480), 32% (472) put an answer here. Those 472 reports average 138, in line with past numbers. But 32% is low enough that we’re pretty vulnerable to selection bias.
(I’ve never taken an IQ test, and left this question blank.)
What if the people who have taken IQ tests are on average smarter than the people who haven’t? My impression is that people mostly take IQ tests when they’re somewhat extreme: either low and trying to qualify for assistive services or high and trying to get “gifted” treatment. If we figure lesswrong draws mostly from the high end, then we should expect the IQ among test-takers to be higher than what we would get if we tested random people who had not previously been tested.
This sounds plausible, but from looking at the data, I don’t think this is happening in our sample. In particular, if this were the case, then we would expect the SAT scores of those who did not submit IQ data to be different from those who did submit IQ data. I ran an Anderson–Darling test on each of the following pairs of distributions:
SAT out of 2400 for those who submitted IQ data (n = 89) vs SAT out of 2400 for those who did not submit IQ data (n = 230)
SAT out of 1600 for those who submitted IQ data (n = 155) vs SAT out of 1600 for those who did not submit IQ data (n = 217)
The p-values came out as 0.477 and 0.436 respectively, which means that the Anderson–Darling test was unable to distinguish between the two distributions in each pair at any significance.
As I did for my last plot, I’ve once again computed for each distribution a kernel density estimate with bootstrapped confidence bands from 999 resamples. From visual inspection, I tend to agree that there is no clear difference between the distributions. The plots should be self-explanatory:
(More details about these plots are available in my previous comment.)
Edit: Updated plots. The kernel density estimates are now fixed-bandwidth using the Sheather–Jones method for bandwidth selection. The density near the right edge is bias-corrected using an ad hoc fix described by whuber on stats.SE.
But 32% is low enough that we’re pretty vulnerable to selection bias
The large majority of LessWrongers in the USA have however also provided their SAT scores, and those are also very high values (from what little I know of SATs)...
The large majority of LessWrongers in the USA have however also provided their SAT scores, and those are also very high values (from what little I know of SATs)...
The reported SAT numbers are very high, but the reported IQ scores are extremely high. The mean reported SAT score, if received on the modern 1600 test, corresponds to an IQ in the upper 120s, not the upper 130s. The mean reported SAT2400 score was 2207, which corresponds to 99th but not 99.5th percentile. 99th percentile is an IQ of 135, which suggests that the self-reports may not be that off compared to the SAT self-reports.
Some of us took the SAT before 1995, so it’s hard to disentangle those scores. A pre-1995 1474 would be at 99.9x percentile, in line with an IQ score around 150-155. If you really want to compare, you should probably assume anyone age 38 or older took the old test and use the recentering adjustment for them.
I’m also not sure how well the SAT distinguishes at the high end. It’s apparently good enough for some high IQ societies, who are willing to use the tests for certification. I was shown my results and I had about 25 points off perfect per question marked wrong. So the distinction between 1475 and 1600 on my test would probably be about 5 total questions. I don’t remember any questions that required reasoning I considered difficult at the time. The difference between my score and one 100 points above or below might say as much about diligence or proofreading as intelligence.
Admittedly, the variance due to non-g factors should mostly cancel in a population the size of this survey, and is likely to be a feature of almost any IQ test.
That said, the 1995 score adjustment would have to be taken into account before using it as a proxy for IQ.
Conversion is a very tricky matter, because the correlation is much less than 1 ( 0.369 in the survey, apparently).
With correlation less than 1, regression towards the mean comes into play, so the predicted IQ from perfect SAT is actually not that high (someone posted coefficients in a parallel discussion), and predicted SAT from very high IQ is likewise not that awesome.
The reason the figures seem rather strange, is that they imply some kind of extreme filtering by IQ here. The negative correlation between time here and IQ suggest that the content is not acting as much of a filter, or is acting as a filter in the opposite direction.
The negative correlation between time here and IQ suggest that the content is not acting as much of a filter, or is acting as a filter in the opposite direction.
Well, alternatively old-timers feel it’s more important to accurately estimate their IQ, and new-comers feel it’s more important to be impressive. There also might not be an effect that needs explaining: I haven’t looked at a scatterplot of IQ by time in community or karma yet for this year; last year, there were a handful of low-karma people who reported massive IQs, and once you removed those outliers the correlation mostly vanished.
You still need to explain how the population ended up so extremely filtered.
Without the rest of the survey, one might imagine that various unusual beliefs here are something that’s only very smart people can see as correct and so only very smart people agree and join, but the survey has shown that said unusual beliefs weren’t correlated with self reported IQ or SAT score.
The Wikipedia article states that those are percentiles of test-takers, not the population as a whole. What percentage of seniors take the SAT? I tried googling, but I could not find the figure.
My first thought is that most people who don’t take the SAT don’t intend to go to college and are likely to be below the mean reported SAT score, but then I realized that a non-negligible subset of those people must have taken only the ACT as their admission exam.
I don’t have solid numbers myself, but percentile of test-takers should underestimate percentile of population. However, there is regression to the mean to take into account, as well as that many people take the SAT multiple times and report the most favorable score, both of which suggest that score on test should overestimate IQ, and I’m fudging it by treating those two as if they cancel out.
Possibly. My suspicion is that less people have taken multiple professional IQ tests (I’ve only taken one professional one) than multiple SATs (I think I took it three times, at various ages). I score significantly better on the Raven’s subtest than on other subtests, and so my IQ.dk score was significantly higher than my professional IQ test last year- but this year I only reported the professional one, because that was all that was asked for. (I might not be representative.)
What if the people who have taken IQ tests are on average smarter than the people who haven’t? My impression is that people mostly take IQ tests when they’re somewhat extreme: either low and trying to qualify for assistive services or high and trying to get “gifted” treatment. If we figure lesswrong draws mostly from the high end, then we should expect the IQ among test-takers to be higher than what we would get if we tested random people who had not previously been tested.
The IQ Question read: “Please give the score you got on your most recent PROFESSIONAL, SCIENTIFIC IQ test—no Internet tests, please! All tests should have the standard average of 100 and stdev of 15.”
Among the subset of people making their data public (n=1480), 32% (472) put an answer here. Those 472 reports average 138, in line with past numbers. But 32% is low enough that we’re pretty vulnerable to selection bias.
(I’ve never taken an IQ test, and left this question blank.)
This sounds plausible, but from looking at the data, I don’t think this is happening in our sample. In particular, if this were the case, then we would expect the SAT scores of those who did not submit IQ data to be different from those who did submit IQ data. I ran an Anderson–Darling test on each of the following pairs of distributions:
SAT out of 2400 for those who submitted IQ data (n = 89) vs SAT out of 2400 for those who did not submit IQ data (n = 230)
SAT out of 1600 for those who submitted IQ data (n = 155) vs SAT out of 1600 for those who did not submit IQ data (n = 217)
The p-values came out as 0.477 and 0.436 respectively, which means that the Anderson–Darling test was unable to distinguish between the two distributions in each pair at any significance.
As I did for my last plot, I’ve once again computed for each distribution a kernel density estimate with bootstrapped confidence bands from 999 resamples. From visual inspection, I tend to agree that there is no clear difference between the distributions. The plots should be self-explanatory:
(More details about these plots are available in my previous comment.)
Edit: Updated plots. The kernel density estimates are now fixed-bandwidth using the Sheather–Jones method for bandwidth selection. The density near the right edge is bias-corrected using an ad hoc fix described by whuber on stats.SE.
Thanks for digging into this! Looks like the selection bias isn’t significant.
The large majority of LessWrongers in the USA have however also provided their SAT scores, and those are also very high values (from what little I know of SATs)...
The reported SAT numbers are very high, but the reported IQ scores are extremely high. The mean reported SAT score, if received on the modern 1600 test, corresponds to an IQ in the upper 120s, not the upper 130s. The mean reported SAT2400 score was 2207, which corresponds to 99th but not 99.5th percentile. 99th percentile is an IQ of 135, which suggests that the self-reports may not be that off compared to the SAT self-reports.
Some of us took the SAT before 1995, so it’s hard to disentangle those scores. A pre-1995 1474 would be at 99.9x percentile, in line with an IQ score around 150-155. If you really want to compare, you should probably assume anyone age 38 or older took the old test and use the recentering adjustment for them.
I’m also not sure how well the SAT distinguishes at the high end. It’s apparently good enough for some high IQ societies, who are willing to use the tests for certification. I was shown my results and I had about 25 points off perfect per question marked wrong. So the distinction between 1475 and 1600 on my test would probably be about 5 total questions. I don’t remember any questions that required reasoning I considered difficult at the time. The difference between my score and one 100 points above or below might say as much about diligence or proofreading as intelligence.
Admittedly, the variance due to non-g factors should mostly cancel in a population the size of this survey, and is likely to be a feature of almost any IQ test.
That said, the 1995 score adjustment would have to be taken into account before using it as a proxy for IQ.
Conversion is a very tricky matter, because the correlation is much less than 1 ( 0.369 in the survey, apparently).
With correlation less than 1, regression towards the mean comes into play, so the predicted IQ from perfect SAT is actually not that high (someone posted coefficients in a parallel discussion), and predicted SAT from very high IQ is likewise not that awesome.
The reason the figures seem rather strange, is that they imply some kind of extreme filtering by IQ here. The negative correlation between time here and IQ suggest that the content is not acting as much of a filter, or is acting as a filter in the opposite direction.
Well, alternatively old-timers feel it’s more important to accurately estimate their IQ, and new-comers feel it’s more important to be impressive. There also might not be an effect that needs explaining: I haven’t looked at a scatterplot of IQ by time in community or karma yet for this year; last year, there were a handful of low-karma people who reported massive IQs, and once you removed those outliers the correlation mostly vanished.
You still need to explain how the population ended up so extremely filtered.
Without the rest of the survey, one might imagine that various unusual beliefs here are something that’s only very smart people can see as correct and so only very smart people agree and join, but the survey has shown that said unusual beliefs weren’t correlated with self reported IQ or SAT score.
The Wikipedia article states that those are percentiles of test-takers, not the population as a whole. What percentage of seniors take the SAT? I tried googling, but I could not find the figure.
My first thought is that most people who don’t take the SAT don’t intend to go to college and are likely to be below the mean reported SAT score, but then I realized that a non-negligible subset of those people must have taken only the ACT as their admission exam.
I don’t have solid numbers myself, but percentile of test-takers should underestimate percentile of population. However, there is regression to the mean to take into account, as well as that many people take the SAT multiple times and report the most favorable score, both of which suggest that score on test should overestimate IQ, and I’m fudging it by treating those two as if they cancel out.
Don’t most people who report IQ scores do the same thing if they have taken multiple tests?
Possibly. My suspicion is that less people have taken multiple professional IQ tests (I’ve only taken one professional one) than multiple SATs (I think I took it three times, at various ages). I score significantly better on the Raven’s subtest than on other subtests, and so my IQ.dk score was significantly higher than my professional IQ test last year- but this year I only reported the professional one, because that was all that was asked for. (I might not be representative.)
Not if they followed the survey instructions, which asked for only the scores from the most recent professional IQ test they took.