Time to abandon cryosleep. I hope this post isn’t too big.
This comparison seems to rely on too many dubious assumptions: First, that the IQ scores reported in the survey were precise for a uniform standard deviation. Second, that these scores correlate strongly with the forms of competence relevant to LessWrong. Third, that this correlation will further correlate strongly with the total Karma of a user. Fourth, it rests on an understanding of the Dunning-Kruger effect and its implications that I either don’t understand or don’t at all agree with.
Pertaining to the question of IQ, I have yet to see a LessWrong survey that required specificity on the IQ question. Standard deviation and test type aren’t included with the answers, and so the answers are hard to standardize. The internal relationship of these scores is obfuscated by us not knowing which tests they were derived from. Yvain’s request to only include “respectable tests” is sensible, but still leaves a lot of room for interpretation, and could reasonably include differing standard deviations. Even assuming a strong prior for the more common standard deviations of 15 and 16, a lot of these IQ scores are out of bounds of what might be considered accurate testing. Tests with a wide battery of subtests will be especially likely to make some scores roof out and lead to distortions of the average—g may be strong, but it’s difficult to combat test design. Don’t expect anything above mensa entry level to be measured especially accurately. 150 is probably indicative of higher ability than 135, but it’s hard to say how strongly a score has been distorted by an arbitrary roof to the level tested.
IQ score totals (when not accounting for standard deviations) are not especially well-correlated to begin with, and not accounting for these variables and many others besides them will only compound that problem. I’m sure there’s interesting stuff you could achieve with the survey numbers, but I doubt an accurate intra-community comparison of the user base is one of those things.
As for the second and third problem, IQ obviously has strong correlations with some forms of competence. However, I would also expect most posters here to be at least vaguely aware of the Dunning-Kruger effect or the general concept it derives from, and so post selectively on the stuff they are fairly sure they know. This would skew the correlations towards widely supported sentiments, well-crafted posts and total volume of posts, except for users who are polymaths of some description (of which, admittedly, we have a few).
As for the fourth problem: if the IQ results are anywhere near accurate, sub-normal ability is very abnormal on LessWrong. Most of us posting here aren’t stupid, or even close to normal intelligence, let alone significantly sub-normal intelligence. The Donning-Kruger effect does not operate on a sliding scale where people of higher intelligence tend to think of themselves as even smarter. It inverts. People of actual ability tend to underestimate themselves. Accounting for that, it is also difficult to quantify the effect on people who are far above the LessWrong mean IQ (if we accept that measurement at all), mainly because those people are very rare. Do they tend to hold extremely pessimistic views of their own ability, or do they estimate more rational than less intelligent individuals? It’s difficult to muster a normative study of something like that—being far out of bounds of any conventional IQ test certainly doesn’t help—and arguing from conjecture would be inaccurate in lieu of some very strong priors.
Time to abandon cryosleep. I hope this post isn’t too big.
This comparison seems to rely on too many dubious assumptions: First, that the IQ scores reported in the survey were precise for a uniform standard deviation. Second, that these scores correlate strongly with the forms of competence relevant to LessWrong. Third, that this correlation will further correlate strongly with the total Karma of a user. Fourth, it rests on an understanding of the Dunning-Kruger effect and its implications that I either don’t understand or don’t at all agree with.
Pertaining to the question of IQ, I have yet to see a LessWrong survey that required specificity on the IQ question. Standard deviation and test type aren’t included with the answers, and so the answers are hard to standardize. The internal relationship of these scores is obfuscated by us not knowing which tests they were derived from. Yvain’s request to only include “respectable tests” is sensible, but still leaves a lot of room for interpretation, and could reasonably include differing standard deviations. Even assuming a strong prior for the more common standard deviations of 15 and 16, a lot of these IQ scores are out of bounds of what might be considered accurate testing. Tests with a wide battery of subtests will be especially likely to make some scores roof out and lead to distortions of the average—g may be strong, but it’s difficult to combat test design. Don’t expect anything above mensa entry level to be measured especially accurately. 150 is probably indicative of higher ability than 135, but it’s hard to say how strongly a score has been distorted by an arbitrary roof to the level tested.
IQ score totals (when not accounting for standard deviations) are not especially well-correlated to begin with, and not accounting for these variables and many others besides them will only compound that problem. I’m sure there’s interesting stuff you could achieve with the survey numbers, but I doubt an accurate intra-community comparison of the user base is one of those things.
As for the second and third problem, IQ obviously has strong correlations with some forms of competence. However, I would also expect most posters here to be at least vaguely aware of the Dunning-Kruger effect or the general concept it derives from, and so post selectively on the stuff they are fairly sure they know. This would skew the correlations towards widely supported sentiments, well-crafted posts and total volume of posts, except for users who are polymaths of some description (of which, admittedly, we have a few).
As for the fourth problem: if the IQ results are anywhere near accurate, sub-normal ability is very abnormal on LessWrong. Most of us posting here aren’t stupid, or even close to normal intelligence, let alone significantly sub-normal intelligence. The Donning-Kruger effect does not operate on a sliding scale where people of higher intelligence tend to think of themselves as even smarter. It inverts. People of actual ability tend to underestimate themselves. Accounting for that, it is also difficult to quantify the effect on people who are far above the LessWrong mean IQ (if we accept that measurement at all), mainly because those people are very rare. Do they tend to hold extremely pessimistic views of their own ability, or do they estimate more rational than less intelligent individuals? It’s difficult to muster a normative study of something like that—being far out of bounds of any conventional IQ test certainly doesn’t help—and arguing from conjecture would be inaccurate in lieu of some very strong priors.
Good to know, thanks.