As Yvain says, “people have been pretty quick to ridicule this survey’s intelligence numbers as completely useless and impossible and so on” because if they’re true, it means that the average LessWronger is gifted. Yvain added a few questions to the 2012 survey, including the ACT and SAT questions and the Myers-Briggs personality type question that I requested (I’ll explain why this is interesting), and that give us a few other things to check against, which has made the figures more believable. The ridicule may be an example of the “virtuous doubt” that Luke warns about in Overconfident Pessimism, so it makes sense to “consider the opposite”:
The distribution of Myers-Briggs personality types on LessWrong replicates the Mensa pattern. This is remarkable since the patterns of personality types here are, in many significant ways, the exact opposite of what you’d find in the regular population. For instance, the introverted rationalists and idealists are each about 1% of the population. Here, they are the majority and it’s the artisans and guardians who are relegated to 1% or less of our population.
Mensa’s personality test results were published in the December 1993 Mensa Bulletin. Their numbers.
So, if you believe that most of the people who took the survey lied about their IQ, you also need to believe all of the following:
That most of these people also realized they needed to do IQ correlation research and fudge their SAT and ACT scores in order for their IQ lie to be believable.
Some explanation as to why the average of lurker’s IQ scores would come out so close to the average of poster’s IQ scores. The lurkers don’t have karma to show off, and there’s no known incentive good enough to get so many lurkers to lie about their IQ score. Vaniver’s figures.
Some explanation for why the personality type pattern at LessWrong is radically different from the norm and yet very similar to the personality type pattern Mensa published and also matched my predictions. Even if they had knowledge of the Mensa personality test results and decided to fudge their personality type responses, too, they somehow managed to fudge them in such a way that their personality types accidentally matched my predictions.
That they decided not to cheat when answering the Bayes birthday question even though they were dishonest enough to lie on the IQ question, motivated to look intelligent, and it takes a lot less effort to fudge the Bayes question than the intelligence and personality questions. (This was suggested by ArisKatsaris).
That both posters and lurkers had some motive strong enough to justify spending 20+ minutes doing the IQ correlation research and fudging personality test questions while probably bored of ticking options after filling out most of a very long survey.
It’s easier just to put the real number in the IQ box than do all that work to make it believable, and it’s not like the liars are likely to get anything out of boasting anonymously, so the cost-benefit ratio is just not working in favor of the liar explanation.
If you think about it in terms of Occam’s razor, what is the better explanation? That most people lied about their IQ, and fudged their SAT, ACT and personality type data to match, or that they’re telling the truth?
Summary of criticism:
Possible Motive to Lie: The desire to be associated with a “gifted” group:
In re to this post, it was argued by NonComposMentis that a potential motive to lie is that if the outside world perceives LessWrong as gifted, then anyone having an account on LessWrong will look high-status. In rebuttal:
I figure that lurkers would not be motivated to fudge their results because they don’t have a bunch of karma on their account to show off and anybody can claim to read LessWrong, so fudging your IQ just to claim that the site you read is full of gifted people isn’t likely to be motivating. I suggested that we compare the average IQs of lurkers and others. Vaniver did the math and they are very, very close..
I argued, among other things, that it would be falling for a Pascal’s mugging to believe that investing the extra time (probably at least $5 worth of time for most of us) into fudging the various different survey questions is likely to contribute to a secret conspiracy to inflate LessWrong’s average IQ.
Did the majority avoid filling out intelligence related questions, letting the gifted skew the results?
Short answer: 74% of people answered at least one intelligence related question and since most people filled out only one or two, the fact that the self-report, ACT and SAT score averages are so similar is remarkable.
I realized, while reading Vaniver’s post that if only 1⁄3 of the survey participants filled out the IQ score, this may have been due to something which could have skewed the results toward the gifted range, for instance, if more gifted people had been given IQ tests for schooling placement (and the others didn’t post their IQ score because they did not know it) or if the amount of pride one has in their IQ score has a significant influence on whether one reported it.
So I went through the data and realized that most of the people who filled out the IQ test question did not fill out all the others. That means that 804 people (74% not 33%) answered at least one intelligence related question. As we have seen, the IQ correlations for the IQ, SAT and ACT questions were very close to each other (unsurprisingly, it looks like something’s up with the internet test… removing those, it’s 63% of survey participants that answered an intelligence related question). It’s remarkable in and of itself that each category of test scores generated an average IQ so similar to the others considering that different people filled them out. I mean if 1⁄3 of the population filled out all of the questions, and the other 2⁄3 filled out none, we could say “maybe the 1⁄3 did IQ correlation research and fudged these” but if most of the population fills out one or two, and the averages for each category come out close to the averages for the other categories, why is that? How would that happen if they were fudging?
It does look to me like people gave whatever test scores they had and that not all the people had test scores to give but it does not look to me like a greater proportion of the gifted people provided an intelligence related survey answer. Instead it looks like most people provided an intelligence related survey answer and the average LessWronger is gifted.
There are a lot of questions on the personality test that have an obvious intelligence component, so it’s possible that people chose the answer they thought was most intelligent.
There are also intelligence related questions where it’s not clear which answer is most intelligent. I listed those.
The intelligence questions would mostly influence the sensing/intuition dichotomy and the thinking/feeling dichotomy. This does not explain why the extraversion/introversion and perceiving/judging results were similar to Mensa’s.
Alternate possibility: The distribution of personality types in Mensa/LW relative to everyone else is an artifact produced by self-identified smart people trying to signal their intelligence by answering ‘yes’ to traits that sound like the traits they ought to have.
eg. I know that a number of the T/F questions are along the lines of “I use logic to make decisions (Y/N)”, which is a no-brainer if you’re trying to signal intelligence.
A hypothetical way to get around this would be to have your partner/family member/best friend next to you as you take the test, ready to call you out when your self-assessment diverges from your actual behaviour (“hold on, what about that time you decided not to go to the concert of [band you love] because you were angry about an unrelated thing?”)
Ok, it’s possible that all of the following happened:
Most of the 1000 people decided to lie about their IQ on the LessWrong survey.
Most of the liars realized that their personality test results were going to be compared with Mensa’s personality type results, and it dawned on them that this would bring their IQ lie into question.
Most of the liars decided that instead of simply skipping the personality test question, or taking it to experience the enjoyment of finding out their type, they were going to fudge the personality test results, too.
Most of the liars actually had the patience to do an additional 72 questions specifically for the purpose of continuing to support a lie when they had just slogged through 100 questions.
Most of the liars did all of that extra work (Researching the IQ correlation with the SAT and the ACT and fudging 72 personality type questions) when it would have been so much easier to put their real IQ in the box, or simply skip the IQ question completely because it is not required.
Most of the liars succeeded in fudging their personality types. This is, of course, possible, but it it is likely to be more complicated than it at first seems. They’d have to be lucky that enough of the questions give away their intelligence correlation in the wording (we haven’t verified that). They’d have to have enough of an understanding of what intelligent people are like that they’d choose the right ones. Questions like these are likely to confuse a non-gifted person trying to guess which answers will make them look gifted:
“You are more interested in a general idea than in the details of its realization”
(Do intelligent people like ideas or details more?)
“Strict observance of the established rules is likely to prevent a good outcome”
(Either could be the smarter answer, depending who you ask.)
“You believe the best decision is one that can be easily changed”
(It’s smart to leave your options open, but it’s also more intellectually self-confident and potentially more rewarding to take a risk based on your decision-making abilities.)
“The process of searching for a solution is more important to you than the solution itself”
(Maybe intelligence makes playing with ideas so enjoyable, gifted people see having the solution as less important.)
“When considering a situation you pay more attention to the current situation and less to a possible sequence of events”
(There are those that would consider either one of these to be the smarter one.)
There were a lot of questions that you could guess are correlated with intelligence on the test, and some of them are no-brainers, but are there enough of those no-brainers with obvious intelligence correlation that a non-gifted person intent on looking as intelligent as possible would be able to successfully fudge their personality type?
The massive fudging didn’t create some totally unexpected personality type pattern. For instance, most people are extraverted. Would they realize the intelligence implications and fudge enough extravert questions to replicate Mensa’s introverted pattern? Would they know to choose the judging questions over the perceiving questions would make them look like Mensans? It makes sense that the thinking vs. feeling and intuiting vs sensing metrics would use questions that would be of the type you’d obviously need to fudge, but why would they also choose introvert and judging answers?
The survey is anonymous and we don’t even know which people gave which IQ responses, let alone are they likely to receive any sort of reward from fudging their IQ score. Can you explain to me:
What reward would most of LessWrong want to get out of lying about their IQs?
Why, in an anonymous context where they can’t even take credit for claiming the IQ score they provided, most of LessWrong is expecting to receive any reward at all?
Can you explain to me why fudged personality type data would match my predictions? Even if they were trying to match them, how would they manage it?
“Lie” is a strawman. One could report an estimate, mis-remember, report the other “IQ” (mental age / chronological age metric), or one may have took any one of entirely faulty online tests that report IQ as high to increase the referral rate (some are bad enough to produce >100 if the answers are filled in at random).
This would be a good point in the event that we were not discussing IQ scores generated by an IQ test selected by Yvain, which many people took at the same time as filling out the survey. This method (and timing) rules out problems due to relying on estimates alone, most of the potential for mis-remembering, (neither of which should be assumed to be likely to result in an average score that’s 30 points too high, as mistakes like these could go in either direction), and, assuming that the IQ test Yvain selected was pretty good, it also rules out the problem of the test being seriously skewed. If you would like to continue this line of argument, one effective method of producing doubt would be to go to the specific IQ test in question, fill out all of the answers randomly, and report the IQ that it produces. If you want to generate a full-on update regarding those particular test results, complete with Yvain being likely to refrain from recommending this source during his next survey, write a script that fills out the test randomly and reports the results so that multiple people can run it and see for themselves what average IQ the test produces after a large number of trials. You may want to check to see whether Yvain or Gwern or someone has already done this before going to the trouble.
Also, there really were people whose concern it was that people were lying on the survey. Your “lie is a strawman” perception appears to have been formed due to not having read the (admittedly massive number of) comments on this.
neither of which should be assumed to be likely to result in an average score that’s 30 points too high, as mistakes like these could go in either direction
Look. People misremember (and remember the largest value, and so on) in the way most favourable to themselves. While mistakes can of course go in either direction, they don’t actually go in either direction. If you ask men to report their penis size (quite literally), they over-estimate; if you ask them to measure, they still overestimate but not by as much. This sort of error is absolutely the norm in any surveys. More so here, as the calibration (on Bayes date of birth question at least) was comparatively very bad.
The situation is anything but symmetric, given that the results are rather far from the mean, on a Gaussian.
Furthermore, given the interest in self improvement, people here are likely to have tried to improve their test scores by practice, which would have considerably lower effect on iqtest.dk unless you practice specifically the Raven’s matrices.
The low scores on iqtest.dk are particularly interesting in light that the scores on the latter are a result of better assignment of priors / processing of probabilities (as, fundamentally, one needs to pick the choice which results in simplest—highest probability—overall pattern. If one is overconfident about the pattern they see being the best, one’s score is lowered, so poor calibration will hurt that test more).
While mistakes can of course go in either direction, they don’t actually go in either direction.
I intuit that this is likely to be a popular view among sceptics, but I do not recall ever being presented with research that supports this by anyone. To avoid the lure of “undiscriminating scepticism”, I am requesting to see the evidence of this.
I agree that, for numerous reasons, self-reported IQ scores, SAT scores, ACT scores and any other scores are likely to have some amount of error, and I think it’s likely for the room for error to be pretty big. On that we agree.
An average thirty points higher than normal seems to me to be quite a lot more than “pretty big”. That’s the difference between an IQ in the normal range and an IQ large enough to qualify for every definition of gifted. To use your metaphor, that’s like having a 6-incher and saying it’s 12. I can see guys unconsciously saying it’s 7 if it’s 6, or maybe even 8. But I have a hard time believing that most of these people have let their imaginations run so far away with them as to accidentally believe that they’re Mensa level gifted when they’re average. I’d bet that there was a significant amount of error, but not an average of 30 points.
If you agree with those two, then whether we agree over all just depends on what specific belief we’re each supporting.
I think these beliefs are supported:
The SAT, ACT, self-reported IQ and / or iqtest.dk scores found on the survey are not likely to be highly accurate.
Despite inaccuracies, it’s very likely that the average LessWrong member has an IQ above average—in other words, I don’t think that the scores reported on the survey are so inaccurate that I should believe that most LessWrongers actually have just an average IQ.
LessWrong is (considering a variety of pieces of evidence, not just the survey) likely to have more gifted people than you’d find by random chance.
Do we agree on those three beliefs?
If not, then please phrase the belief(s) you want to support.
Even if every self-reported IQ is exactly correct, the average of the self-reported IQ values can still be (and likely will still be) higher than the average of the readership’s IQ values.
Consider two readers, Tom and Jim. Tom does an IQ test, and gets a result of 110. Jim does an IQ test, and gets a result of 90. Tom and Jim are both given the option to fill in a survey, which asks (among other questions) what their IQ is. Neither Tom nor Jim intend to lie.
However, Jim seems significantly more likely to decide not to participate; while Tom may decide to fill in the survey as a minor sort of showing off. This effect will skew the average upwards. Perhaps not 30 points upwards… but it’s an additional source of bias, independent of any bias in individual reported values.
I remember looking into this when I looked at the survey data. There were only a handful of people who reported two-digit IQs, which is consistent with both the concealment hypothesis and the high average intelligence hypothesis. If you assume that nonresponders have an IQ of 100 on average the average IQ across everyone drops down to 112. (I think this is assumption is mostly useful for demonstrative purposes; I suspect that the prevalence of people with two-digit IQs on LW is lower than in the general population.)
(You could do some more complicated stuff if you had a functional form for concealment that you wanted to predict, but it’s not obvious to me that IQs on LW actually follow a normal distribution, which would make it hard to separate out the oddities of concealment with the oddities of the LW population.)
Select a random sampling of people (such as by picking names from the phonebook). Ask each person whether they would like to fill in a survey which asks, among other things, for their IQ. If a sufficiently large, representative sample is taken, the average IQ of the sample is likely to be 100 (confirm if possible). Compare this to the average reported IQ, in order to get an idea of the size of the bias.
Select a random sampling of lesswrongers, and ask them for their IQs. If they all respond, this should cut out the self-selection bias (though the odds are that at least some of them won’t respond, putting us back at square one).
As one of the sceptics, I might as well mention a specific feature of the self-reported IQs that made me pretty sure they’re inflated. (Even before I noticed this feature, I expected the IQs to be inflated because, well, they’re self-reported. Note that I’m not saying people must be consciously lying, though I wouldn’t rule it out. Also, I agree with your three bullet points but still find an average LW IQ of 138-139 implausibly high.)
The survey has data on education level as well as IQ. Education level correlates well with IQ, so if the self-reported IQ & education data are accurate, the subsample of LWers who reported having a “high school” level of education (or less) should have a much lower average IQ. But in fact the mean IQ of the 34% of LWers with a high school education or less was 136.5, only 2.2 points less than the overall mean.
There is a pretty obvious bias in that calculation: a lot of LWers are young and haven’t had time to complete their education, however high their IQs. This stacks the deck in my favour because it means the high-school-or-less group includes a lot of people who are going to get degrees but haven’t yet, which could exaggerate the IQ of the high-school-or-less group.
I can account for this bias by looking only at the people who said they were ≥29 years old. Among that older group, only 13% had a high school education or less...but the mean IQ of that 13% was even higher* at 139.6, almost equal to the mean IQ of 140.0 for older LWers in general. The sample sizes aren’t huge but I think they’re too big to explain this near-equality away as statistical noise. So IQ or education level or age was systematically misreported, and the most likely candidate is IQ, ’cause almost everyone knows their age & education level, and nerds probably have more incentive to lie on a survey about their IQ than about their age or education level.
* Assuming people start university at age 18, take 3 years to get a bachelor’s, a year to get a master’s, and then up to 7 years to get a PhD, everyone who’s going to get a PhD will have one at age 29. In reality there’re a few laggards but not enough to make much difference; basically the same result comes out if I use age 30 or age 35 as a cutoff.
I can account for this bias by looking only at the people who said they were ≥29 years old.* Among that older group, only 13% had a high school education or less...but the mean IQ of that 13% was even higher at 139.6, almost equal to the mean IQ of 140.0 for older LWers in general.
And I suspect if you look at the American population for that age cohort, you’ll find a lot higher a percentage than 13% which have a “high school education or less”… All you’ve shown is that of the highschool-educated populace, LW attracts the most intelligent end, the people who are the dropouts for whatever reason. Which for high-IQ people is not that uncommon (and one reason the generic education/IQ correlation isn’t close to unity). LW filters for IQ and so only smart highschool dropouts bother to hang out here? Hardly a daring or special pleading sort of suggestion. And if we take your reasoning at face-value that the general population-wide IQ/education correlate must hold here, it would suggest that there would be hardly any autodidacts on LW (clearly not the case), such as our leading ‘high school education or less’ member, Eliezer Yudkowsky.
All you’ve shown is that of the highschool-educated populace, LW attracts the most intelligent end, the people who are the dropouts for whatever reason.
Right, but even among LWers I’d still expect the dropouts to have a lower average IQ if all that’s going on here is selection by IQ. Sketch the diagram. Put down an x-axis (representing education) and a y-axis (IQ). Put a big slanted ellipse over the x-axis to represent everyone aged 29+.
Now (crudely, granted) model the selection by IQ by cutting horizontally through the ellipse somewhere above its centroid. Then split the sample that’s above the horizontal line by drawing a vertical line. That’s the boundary between the high-school-or-less group and everyone else. Forget about everyone below the horizontal line because they’re winnowed out. That leaves group A (the high-IQ people with less education) and group B (the high-IQ people with more).
Even with the filtering, group A is visibly going to have a lower average IQ than B. So even though A comprises “the most intelligent end” of the less educated group, there remains a lingering correlation between education level and IQ in the high-IQ sample; A scores less than B. The correlation won’t be as strong as the general population-wide correlation you refer to, but an attenuated correlation is still a correlation.
Education level correlates well with IQ, so if the self-reported IQ & education data are accurate, the subsample of LWers who reported having a “high school” level of education (or less) should have a much lower average IQ.
It seems implausible to me that education level would screen off the same parts of the IQ distribution in LW as it does in the general population, at least at its lower levels. It’s not too unreasonable to expect LWers with PhDs to have higher IQs than the local mean, but anyone dropping out of high school or declining to enter college because they dislike intellectual pursuits, say, seems quite unlikely to appreciate what we tend to talk about here.
It’s not too unreasonable to expect LWers with PhDs to have higher IQs than the local mean,
Upvoted. If I repeat the exercise for the PhD holders, I find they have a mean IQ of 146.5 in the older subsample, compared to 140.0 for the whole older subsample, which is consistent with what you wrote.
I did a back-of-the-R-session guesstimate before I posted and got a two-tailed p-value of roughly 0.1, so not significant by the usual standard, but I figured that was suggestive enough.
Doing it properly, I should really compare the PhD holders’ IQ to the IQ of the non-PhD holders (so the samples are disjoint). Of the survey responses that reported an IQ score and an age of 29+, 13 were from people with PhDs (mean IQ 146.5, SD 14.8) and 135 were from people without (mean IQ 139.3, SD 14.3). Doing a t-test I get t = 1.68 with 14.2 degrees of freedom, giving p = 0.115.
It’s a third of a SD and change (assuming a 15-point SD, which is the modern standard), which isn’t too shabby; comparable, for example, with the IQ difference between managerial and professional workers. Much smaller than the difference between the general population and PhDs within it, though; that’s around 25 points.
Even before I noticed this feature, I expected the IQs to be inflated because, well, they’re self-reported
Yes, and even without particular expectation of inflation, once you see IQs that are very high, you can be quite sure IQs tend to be inflated simply because of the prior being the bell curve.
Any time I see “undiscriminating scepticism” mentioned, it’s a plea to simply ignore necessarily low priors when evidence is too weak to change conclusions. Of course, it’s not true “undiscriminating scepticism”. If LW undergone psychologist-administered IQ testing and that were the results, and then there was a lot of scepticism, you could claim that there’s some excessive scepticism. But as it is, rational processing of probabilities is not going to discriminate that much based on self reported data.
I intuit that this is likely to be a popular view among sceptics,
Sceptics in that case, I suppose, being anyone who actually does the most basic “Bayesian” reasoning, such as starting with a Gaussian prior when you should (and understanding how an imperfect correlation between self reported IQ and actual IQ would work on that prior, i.e. regression towards the mean when you are measuring by proxy). I picture there’s a certain level of Dunning Kruger effect at play, whereby those least capable of probabilistic reasoning would think themselves most capable (further evidenced by calibration; even though the question may have been to blame, I’m pretty sure most people believed that a bad question couldn’t have that much of an impact).
but I do not recall ever being presented with research that supports this by anyone.
Wikipedia to the rescue, given that a lot of stuff is behind the paywall...
“The disparity between actual IQ and perceived IQ has also been noted between genders by British psychologist Adrian Furnham, in whose work there was a suggestion that, on average, men are more likely to overestimate their intelligence by 5 points, while women are more likely to underestimate their IQ by a similar margin.”
Just about any internet forum would select for people owning a computer and having an internet connection and thus cut off the poor, mentally disabled, and so on, improving the average. So when you state it this way—mere “above average”—it is a set of completely unremarkable beliefs.
It’d be interesting to check how common are advanced degrees among white Americans with actual IQ of 138 and above, but I can’t find any info.
Sceptics in that case, I suppose, being anyone who actually does the most basic “Bayesian” reasoning, such as starting with a Gaussian prior when you should (and understanding how an imperfect correlation between self reported IQ and actual IQ would work on that prior, i.e. regression towards the mean when you are measuring by proxy).
This was one of the things I checked when I looked into the IQ results from the survey here and here. One of the things I thought was particularly interesting was that there was a positive correlation between self-reported IQ and iqtest.dk (which is still self-reported, and could have been lied on, but hopefully this is only deliberate lies, rather than fuzzy memory effects) among posters and a negative correlation among lurkers. This comment might also be interesting.
I endorse Epiphany’s three potential explanations, and would quantify the last one: I strongly suspect the average IQ of LWers is at least one standard deviation above the norm. I would be skeptical of the claim that it’s two standard deviations above the norm, given the data we have.
Wow, that’s quite interesting—that’s some serious Dunning-Kruger. Scatterplot could be of interest.
Thing to keep in mind is that even given a prior that errors can go either way equally, when you have obtained a result far from the mean, you must expect that errors (including systematic errors) were predominantly in that direction.
Other issue is that in a 1000 people, about 1 will have an IQ of >=146 or so , while something around 10 will have fairly severe narcissism (and this is not just your garden variety of overestimating oneself, but the level where it interferes with normal functioning).
Self reported IQ of 146 is thus not really a good sign overall. Interestingly some people do not understand that and go on how the others “punish” them for making poorly supported statements of exceptionality, while it is merely a matter of correct probabilistic reasoning.
I endorse Epiphany’s three potential explanations, and would quantify the last one: I strongly suspect the average IQ of LWers is at least one standard deviation above the norm. I would be skeptical of the claim that it’s two standard deviations above the norm, given the data we have.
The actual data is linked in the post near the end. If you drop three of the lurkers- who self-reported 180, 162, and 156 but scored 102, 108, and 107- then the correlation is positive (but small). (Both samples look like trapezoids, which is kind of interesting, but might be explained by people using different standard deviations.)
something around [1 in] 10 will have fairly severe narcissism (and this is not just your garden variety of overestimating oneself, but the level where it interferes with normal functioning).
That sounds pretty high to me. I haven’t looked into narcissism as such, but I remember seeing similar numbers for antisocial personality disorder when I was looking into that, which surprised me; the confusion went away, however, when I noticed that I was looking at the prevalence in therapy rather than the general population.
You know, people do lie to themselves. It’s a sad but true (and well known around here) fact about human psychology that humans have surprisingly bad models of themselves. It is simply true that if you asked a bunch of people selected at random about their (self-reported) IQ scores, you would get an average of more than 100. One would hope that LessWrongers are good enough at detecting bias in order to mostly dodge that bullet, but the evidence of whether or not we actually are that good at it is scarce at best.
Your unintentional lie explanation does not explain how the SAT scores ended up so closely synchronised to the IQ scores—as we know, one common sign of a lie is that the details do not add up. Synchronising one’s SAT scores to the same level as one’s IQ scores would most likely require conscious effort, making the discrepancy obvious to the LessWrong members who took the survey. If you would argue that they were likely to have chosen corresponding SAT scores in some way that did not require them to become consciously aware of discrepancies in order to synchronize the scores, how would you support the argument that they synched them on accident? If not, then would you support the argument that LessWrong members consciously lied about it?
Linda Silverman, a giftedness researcher, has observed that parents are actually pretty decent at assessing their child’s intellectual abilities despite the obvious cause for bias.
“In this study, 84% of the children whose parents indicated that they fit three-fourths of the characteristics tested above 120 IQ. ” (An unpublished study, unfortunately.)
This isn’t exactly the same as managing knowledge of one’s own intellectual abilities, but if it would seem to you that parents would most likely be hideously biased when assessing their children’s intellectual abilities even though, according to a giftedness researcher, this is probably not the case, then should you probably also consider that your concern that most LessWrong members are likely to subconsciously falsify their own IQ scores by a whopping 30 points (if that is your perception) may be far less likely to be a problem than you thought?
That most people lied about their IQ, and fudged their SAT, ACT and personality type data to match, or that they’re telling the truth?
Scores on standardized tests like SAT and ACT can be improved via hard work and lots of practice—there are abundant practice books out there for such tests. It is entirely conceivable that those self-reported IQs were generated via comparing scores on these standardized tests against IQ-conversion charts. I.e., with very hard work, the apparent IQs are in the 130+ range according to these standardised tests; but when it comes to tests that measure your native intelligence (e.g., iqtest.dk), the scores are significantly lower. In future years, it would be advisable for the questionnaire to ask participants how much time they spent in total to prepare for tests such as SAT and ACT—and even then you might not get honest answers. That brings me to the point of lying...
it’s not like the liars are likely to get anything out of boasting anonymously
Not necessarily true. If the survey results show that LWers generally have IQs in the gifted range, then it allows LWers to signal their intelligence to others just by identifying themselves as LWers. People would assume that you probably have an IQ in the gifted range if you tell them that you read LW. In this case, everyone has an incentive to fudge the numbers.
erratio has also pointed out that participants might have answered those personality tests untruthfully in order to signal intelligence, so I shan’t belabour the point here.
People would assume that you probably have an IQ in the gifted range if you tell them that you read LW. In this case, everyone has an incentive to fudge the numbers.
Ok, now here is a motive! I still find it difficult to believe that:
Most of 1000 people care so much about status that they’re willing to prioritize it over truth, especially since this is LessWrong where we gather around the theme of rationality. If there’s anyplace you’d think it would be unlikely to find a lot of people lying about things on a survey, it’s here.
The people who take the survey know that their IQ contribution is going to be watered down by the 1000 other people taking the survey. Unless they have collaborated by PM and have made a pact to fudge their IQ test figures, these frequently math oriented people must know that fudging their IQ figure is going to have very, very little impact on the average that Yvain calculates. I do not know why they’d see the extra work as worthwhile considering the expected amount of impact. Thinking that fudging only one of the IQs is going to be worthwhile is essentially falling for a Pascal’s mugging.
Registration at LessWrong is free and it’s not exclusive. At all. How likely is it, do you think, that this group of rationality-loving people has reasoned that claiming to have joined a group that anybody can join is a good way to brag about their awesomeness?
I suppose you can argue that people who have karma on their accounts can point to that and say “I got karma in a gifted group” but lurkers don’t have that incentive. All lurkers can say is “I read LessWrong.” but that is harder to prove and even less meaningful than “I joined LessWrong”.
Putting the numbers where our mouths are:
If the average IQ for lurkers / people with low karma on LessWrong is pretty close to the average IQ for posters and/or people with karma on LessWrong, would you say that the likelihood of post-making/karma-bearing LessWrongers lying on the survey in order to increase other’s status perceptions of them is pretty low?
Do you want to get these numbers? I’ll probably get them later if you don’t, but I have a pile of LW messages and a bunch of projects going on right now so there will be a delay and a chance that I completely forget.
165 out of 549 responses without reported positive karma (30%) self-reported an IQ score; the average response was 138.44.
181 out of 518 responses with reported positive karma (34%) self-reported an IQ score; the average response was 138.25.
One of the curious features of the self-reports is how many of the IQs are divisible by 5. Among lurkers, we had 2 151s, 1 149, and 10 150s.
I think the average self-response is basically worthless, since it’s only a third of responders and they’re likely to be wildly optimistic.
So, what about the Raven’s test? In total, 188 responders with positive karma (36%), and 164 responders without positive karma (30%) took the Raven’s test, with averages of 126.9 and 124.4. Noteworthy is the new max and min- the highest scorer on the Raven’s test claimed to get 150, and the three sub-100 scores were 3, 18, and 66 (of which I suspect only the last isn’t a typo or error of some sort).
Only 121 users both self-reported IQ and took the Raven’s test. The correlation between their mean-adjusted self-reported IQ and mean-adjusted Raven’s test was an abysmal .2. Among posters with positive karma, the correlation was .45; among posters without positive karma, the correlation was -.11.
Thank you for these numbers, Vaniver! I should have thanked you sooner. I had become quite busy (partly with preparing my new endless September post) so I did not show up to thank you promptly. Sorry about that.
The people who take the survey know that their IQ contribution is going to be watered down by the 1000 other people taking the survey.
I have thought of that. But a person who wants to lie about his IQ would think this way: If I lie and other LWers do not, it is true that my impact on the average calculated IQ will be negligible, but at least it will not be negative; but if I lie and most other LWers also lie, then the collective upward bias will lead to a very positive result which would portray me in a good light when I associate myself with other LWers. So there is really no incentive to not lie.
(I’m not saying that they definitely lied; I’m merely pointing out that this is something to think about.)
How likely is it, do you think, that this group of rationality-loving people has reasoned that claiming to have joined a group that anybody can join is a good way to brag about their awesomeness?
Fair point; but very often the kind of clubs you join does indicate something about your personality and interests, regardless of whether you are actually an active/contributing member or not. Saying “I read LessWrong” or “I joined LessWrong” certainly signals to me that you are more intelligent than someone who joined, say, Justin Bieber’s fan club, or the Twilight fan-fiction club. And if there are numbers showing that LW readers tend to have IQs in the gifted range, naturally I would think that X is probably quite intelligent just by virtue of the fact that X reads LW.
One last point is that LWers might not be deliberately lying: Perhaps they were merely victim to the Dunning-Kruger effect when self-reporting IQs. I am not sure if there are any studies showing that intelligent people are generally less likely to fall prey to the Dunning-Kruger effect.
Last but not least, I would again like to suggest that future surveys include questions asking people how much time they spent on average preparing for exams such as the SAT and the ACT—as I pointed out previously, scores on such exams can be very significantly improved just by studying hard, whereas tests like iqtest.dk actually measure your native intelligence.
Not true. It would probably take at least 20 minutes to fudge all the stuff that has to be fudged. When you’re already fatigued from filling out survey questions, that’s even less desirable at that time. At best, this would be falling for a Pascal’s mugging. True that some people may. But would the majority of survey participants… at a site about rationality?
Perhaps they were merely victim to the Dunning-Kruger effect when self-reporting IQs
They were not asked to assess their own IQ they were asked to report the results of a real assessment. To report something other than the results of a real assessment is a type of lie in this case.
I would again like to suggest that future surveys include questions asking people how much time they spent on average preparing for exams
That’s a suggestion for Yvain. I don’t assist with the surveys.
(I believe Mensa’s personality test results were published in the December 2006 Mensa newsletter which is, unfortunately, behind a login on the Mensa website, so I can’t link to it here.)
Make a copy and post it. Most browsers have the ability to print/save pages as PDFs or various forms of HTML.
This also explains a lot of things. People regard IQ as if it is meaningless, just a number, and they often get defensive when intellectual differences are acknowledged. I spent a lot of time doing research on adult giftedness (though I’m most interested in highly gifted+ adults) and, assuming the studies were done in a way that is useful (I’ve heard there are problems with this), and my personal experiences talking to gifted adults are halfway decent as representations of the gifted adult population, there are a plethora of differences that gifted adults have. For instance, in “You’re Calling Who A Cult Leader?” Eliezer is annoyed with the fact that people assume that high praise is automatic evidence that a person has joined a cult. What he doesn’t touch on is that there are very significant neurological differences between people in just about every way you could think of, including emotional excitability. People assume that others are like themselves, and this causes all manner of confusion. Eliezer is clearly gifted and intense and he probably experiences admiration with a higher level of emotional intensity than most. If the readers of LessWrong and Hacker News are gifted, same goes for many of them. To those who feel so strongly, excited praise may seem fairly normal. To all those who do not, it probably looks crazy. I explained more about excitability in the comments.
I also want to say (without getting into the insane amount of detail it would take to justify this to the LW crowd—maybe I will do that later, but one bit at a time) that in my opinion, as a person who has done lots of reading about giftedness and has a lot of experience interacting with gifted people and detecting giftedness, the idea that most survey respondents are giving real answers on the IQ portion of the survey seems very likely to me. I feel 99% sure that LessWrong’s average IQ really is in the gifted range, and I’d even say I’m 90%+ sure that the ballpark hit on by the surveys is right. (In other words, they don’t seem like a group of predominantly exceptionally or profoundly gifted Einsteins or Stephen Hawkings, or just talented people at the upper ends of the normal range with IQs near 115, but that an average IQ in the 130′s / 140′s range does seem appropriate.)
This says nothing about the future though… The average IQ has been decreasing on each survey for an average of about two points per year. If the trend continues, then in as many years as LessWrong has been around, LessWrong may trend so far toward the mean that LessWrong will not be gifted anymore (by all IQ standards that is, it would still be gifted by some definitions and IQ standards but not others). I will be writing a post about the future of LessWrong very soon.
Eliezer is clearly gifted and intense and he probably experiences admiration with a higher level of emotional intensity than most. If the readers of LessWrong and Hacker News are gifted, same goes for many of them. To those who feel so strongly, excited praise may seem fairly normal. To all those who do not, it probably looks crazy.
Would you predict then that people who’re not gifted are in general markedly less inclined to praise things with a high level of intensity?
This seems to me to be falsified by everyday experience. See fan reactions to Twilight, for a ready-to-hand example.
My hypothesis would simply be that different people experience emotional intensity as a reaction to different things. Thus, some think we are crazy and cultish, while also totally weird for getting excited about boring and dry things like math and rationality… while some of us think that certain people who are really interested in the lives of celebrities are crazy and shallow, while also totally weird for getting excited about boring and bad things like Twilight.
This also leads each group to think that the other doesn’t get similar levels of emotional intensity, because only the group’s own type of “emotional intensity” is classified as valid intensity and the other group’s intensity is classified as madness, if it’s recognized at all. I’ve certainly made the mistake of assuming that other people must live boring and uninteresting lives, simply because I didn’t realize that they genuinely felt very strongly about the things that I considered boring. (Obligatory link.)
(Of course, I’m not denying there being variation in the “emotional intensity” trait in general, but I haven’t seen anything to suggest that the median of this trait would be considerably different in gifted and non-gifted populations.)
If you want to find them in person, the latest Twilight movie is still in theaters, although you’ve missed the people who made a point of seeing it on the day of the premier.
Haha, I guess so. I am very, very nerdy. I had fun getting worldly in my teens and early 20′s, but I’ve learned that most people alienate me, so I’ve isolated myself into as much of an “ivory tower” as possible. (Which consists of me doing things like getting on my computer Saturday evenings and nerding so hard that I forget to eat.)
If you want to find them in person...
Not really.
the latest Twilight movie is still in theaters, although you’ve missed the people who made a point of seeing it on the day of the premier.
What did they do when you saw them?
How do we distinguish the difference between the kind of fanaticism that mentally unbalanced people display for, say, a show that is considered by many to have unhealthy themes and the kind of excitement that normal people display for the things they love? Maybe Twilight isn’t the best example here.
I didn’t. I don’t particularly have to go out of my way to find Twilight fans, but if I did, I wouldn’t.
How do we distinguish the difference between the kind of fanaticism that mentally unbalanced people display for, say, a show that is considered by many to have unhealthy themes and the kind of excitement that normal people display for the things they love? Maybe Twilight isn’t the best example here.
I think you’re dramatically overestimating the degree to which fans of Twilight are psychologically abnormal. Harlequin romance was already an incredibly popular genre known for having unhealthy themes. Twilight, like Eragon, is a mostly typical work of its genre with a few distinguishing factors which sufficed to garner it extra attention, which expanded to the point of explosive popularity as it started drawing in people who weren’t already regular consumers of the genre.
I think you’re dramatically overestimating the degree to which fans of Twilight are psychologically abnormal.
I wouldn’t be surprised if this is true.
This still does not answer the question “What sample can we use that filters out fanaticism from mentally unbalanced people to compare the type of excitement that gifted people feel to the type of excitement that everyone else feels?” Not to assume that no gifted people are mentally unbalanced… I suppose we’d really have to filter those out of both groups.
How do we distinguish the difference between the kind of fanaticism that mentally unbalanced people display for, say, a show that is considered by many to have unhealthy themes and the kind of excitement that normal people display for the things they love?
It’s true that the downward trend can’t go on forever, and to say that it’s definitely going to continue would be (all by itself, without some other arguments) an appeal to history or slippery slope fallacy. However, when we see a trend as consistent and as potentially meaningful as the one below, it makes sense to start wondering why it is happening:
I was mostly just trying to point out that you are extrapolating from a sample size of three points. Three points which have a tremendous amount of common causes that could explain the variation. Furthermore you aren’t extrapolating 10% further from the span of your data, which might be ok, but actually 100% further. You’re extrapolating for as long as we have data, which is… absurd.
One, I am used to seeing the term “sample size” applied to things like the people being studied, not a number of points done in a calculation. If there is some valid use of the term “sample size” that I am not aware of would you mind pointing me in the correct direction?
Two, I am not sure where you’re getting “three points” from. If you mean the amount of IQ points that LessWrong has lost on the studies, then it was 7.18 points, not three.
Three points which have a tremendous amount of common causes that could explain the variation.
Two points per year, which could be explained in other ways, sure. No matter what the trend, it could be explained in other ways. Even if it was ten points per year we could still say something like “The smartest people got bored taking the same survey over and over and stopped.” There are always multiple ways to explain data. That possibility of other explanations does not rule out the potential that LessWrong is losing intelligent people.
Furthermore you aren’t extrapolating 10% further from the span of your data, which might be ok, but actually 100% further.
Not sure what these 10% and 100% figures correspond to. If I am to understand why you said that, you will have to be specific about what you mean.
You’re extrapolating for as long as we have data, which is… absurd.
Including all of the data rather than just a piece of the data is bad why?
Three points referred to the number of surveys taken, which I didn’t bother to look up, but I believe is three.
10% and 100% referred to the time span over which these data points referred to, ie. three years. Basically, I might be OK with you making a prediction for the next three months (still probably not) but extrapolating for three years based on three years of data seems a bit much to me.
Oh I see. The problem here is that “if the trend continues” is not a prediction. “I predict the trend will continue” would be a prediction. Please read more carefully the next time. You confused me quite a bit.
If you’re not making a prediction, then it’s about as helpful as saying “If the moon crashes into North America next year, LW communities will largely cease to exist.”
Looks like Aumann at work. My own readings, though more specifically on teenage giftedness in the 145+ range, along with stuff on ASD and asperger, heavily corroborate with this.
When I was 17, my (direct) family and I had strong suspicions that I was in this range of giftedness—suspicions which were never reliably tested, and thus neither confirmed nor infirmed. It’s still up in the air and I still don’t know whether I fit into some category of gifted or special individuals, but at some point I realized that it wasn’t all that important and that I just didn’t care.
I might have to explore the question a bit more in depth if I decide to return into the official educational system at some point (I mean, having a paper certifying that you’re a genius would presumably kind of help when making a pitch at university to let you in without the prerequisite college credit because you already know the material). Just mentioning all of the above to explain a bit where my data comes from. Both of my parents and myself were all reading tons of books, references, papers and other information along with several interviews with various psychology professionals for around three months.
Also, and this may be another relevant point, the only recognized, official IQ test I ever took was during that time, and I had a score of “above 130”² (verbal statement) and reportedly placed in the 98th and 99th percentiles on the two sections of a modified WAIS test. The actual normalized score was not included in the report (that psychologist(?¹) sucked, and also probably couldn’t do the statistics involved correctly in the first place).
However, I was warned that the test lost statistical significance / representativeness / whatever above 125, and so that even if I had an IQ of 170+ that test wouldn’t have been able to tell—it had been calibrated for mentally deficient teenagers and very low IQ scores (and was only a one-hour test, and only ten of the questions were written, the rest dynamic or verbal with the psychologist). Later looking-up-stats-online also revealed that the test result distributions were slightly skewed, and that a resulting converted “IQ” of “130″ on this particular test was probably more rare in the general population than an IQ of 130 normally represents, because of some statistical effects I didn’t understand at the time and thus don’t remember at all.
Where I’m going with this is that this doesn’t seem like an isolated effect at all. In fact, it seems like most of North America in general pays way more attention to mentally deficient people and low IQs than to high-IQs and gifted individuals. Based on this, I have a pretty high current prior that many on LW will have received scores suffering from similar effects if they didn’t specifically seek the sorts of tests recommended by Mensa or the likes, and perhaps even then.
Based on this, I would expect such effects to compensate or even overcompensate for any upward nudging in the self-reporting.
=====
I don’t know if it was actually a consulting psychologist. I don’t remember the title she had (and it was all done in French). She was “officially” recognized to be in legal capacity to administrate IQ tests in Canada, though, so whatever title is normally in charge of that is probably the right one.
Based on this, the other hints I mention in the text, and internet-based IQ tests consistently giving me 150-ish numbers when at peak performance and 135-ish when tired (I took those a bit later on, perhaps six months after the official one), 135 is the IQ I generally report (including in the LW survey) when answering forms that ask for it and seems like a fairly accurate guess in terms of how I usually interact with people of various IQ levels.
Was Mensa’s test conducted on the internet? The internet has a systematic bias in personalities.
For example, reddit subscriptions to each personality type reddit favor Introversion and Intuition
4,828 INTJ
4,457 INTP
1,817 INFP
1,531 INFJ
IAWYC, but “the internet” is way too broad for what you actually mean—ISTM that a supermajority of teenagers and young adults in developed countries uses it daily, though plenty of them mostly use it for Facebook, YouTube and similar and probably have never heard of Reddit. (Even I never use Reddit unless I’m following a link to a particular thread from somewhere else—but the first letter of my MBTI is E so this kind of confirms your point.)
Yeah...by “internet” what I meant was sites that most people do not know about—sites that you would only stumble upon in the course of extensive net usage. I once described it to a friend as “deep” vs “shallow” internet, with depth corresponding to the extent to which a typical visitor to the website uses the internet. Even within a website (say reddit) a smaller sub-reddit would be “deeper” than a main one.
I’m myself am actually a counterexample to my own “extroverts don’t use the internet as much” notion...but I’m only a moderate extrovert. (ENTP or ENFP depending on the test...ENTP description fits better. I listed ENTP in the survey.)
Yeah...by “internet” what I meant was sites that most people do not know about—sites that you would only stumble upon in the course of extensive net usage. I once described it to a friend as “deep” vs “shallow” internet, with depth corresponding to the extent to which a typical visitor to the website uses the internet. Even within a website (say reddit) a smaller sub-reddit would be “deeper” than a main one.
By that definition, there are many nearly disconnected “deep internets”.
Yes...i’m confused. Is this supposed to be a flaw in the definition? The idea here is to use relative obscurity to describe the degree to which a site is visited only by Internet users who do heavy exploring. There are only a few “shallow” regions… Facebook, Wikipedia, twitter...the shallowest being google. These are all high traffic and even people who never use computers have heard some of these words. There are many deep regions, on the other hand, and most are disconnected.
Yeah, different websites have different personality skews, which complicates things. Fortunately there’s evidence against Mensa having used an online sample: Epiphany said the results were published in December 1993. It’s fairly easy to give a survey to an Internet forum nowadays, but where would Mensa have found an online sample back in ’93? IRC? Usenet? (There is a rec.org.mensa where people posted about personality and the Myers-Briggs back in 1993, but the only relevant post that year was someone asking about Mensans’ personalities to no avail.)
To suggest that people on the internet may have certain personality types is a good suggestion, but it raises two questions:
Might your example of Reddit be similar to LW because LW gets lots of users from Reddit? (Or put another way, if the average LessWronger is gifted, maybe “the apple doesn’t fall far from the tree” and Reddit has lots of gifted people, too.)
Might gifted people gather in large numbers on the internet because it’s easier to find people with similar interests? (Just because people on the internet tend to have those personality types, it doesn’t mean they’re not gifted.)
As for “the internet” having a systematic bias in personalities, I would like to see the evidence of this that’s not based on a biased sample. It’s likely that the places you go to find people like you will, well, have people like you, so even if you (or somebody else on one of those sites) observed a pattern in personality types across sites they hang out on, the sample is likely to be biased.
I’d say “LW has about as many gifted people as Reddit (proportionally)” should be a sort of null hypothesis: if this is true, then people on LessWrong are not actually surprisingly smart.
I wouldn’t say that’s a reasonable null. Reddit has like 8 million users; 2% of the 310m American population is just 6.2m, so it would be difficult for Reddit to be 100% gifted while LW could easily be. The size disparity is so large that such a null seems more than a little weird.
I don’t think I understand your objection. If LW were 100% gifted (while Reddit, presumably, is not?) wouldn’t that be evidence that there’s some sort of IQ selection at work? (or, conceivably, that just being on LW makes people smarter, although I think that’s not supposed to be a thing).
I’m saying that we could, just from knowing how big Reddit is, reject out of hand all sorts of proportions of gifted because it would be nigh impossible; a set of nulls (the proportions 0-100%), many of which (all >75%) we can reject before collecting any data is a pretty strange choice to make!
Well, really what I want to ask is: is LW any different, IQ-wise, from a random selection of Redditors of the same size? Possibly stating it in terms of a proportion of “gifted” people is misleading, but that’s not as interesting anyway.
Well, I don’t see the difference either, but I’m still not entirely sure what about this hypothesis seems unreasonable to you, so I was hoping this reformulation would help.
The reasoning behind it is as follows: I figure a generic discussion board on the Internet has roughly the same IQ distribution as Reddit. If LW has a high average IQ, but so does Reddit, then presumably these are both due to the selection effect of “someone who posts on an online discussion board”. So to see if LW is genuinely smarter, we should be comparing it to Reddit, not to the Normal(100,15) distribution.
Okay, fair enough. I don’t actually have much experience with Reddit.
I still think it’s a reasonable reference class. For one thing, LW runs on Reddit-based code. In particular, I would say that being significantly smarter than Reddit is a good cutoff for the feeling of smugness to start kicking in.
Maybe it just means Reddit-folk are surprisingly smart? I mean, IQ 130 corresponds to 98th percentile. The usual standard for surprise is 95th percentile.
That’s a good point—I hadn’t considered sample bias. Extending that point, though, Lesswrong and Mensa are a biased sample in more than the simple fact that the people are gifted. It is only a subset of gifted people that choose to participate in Mensa It should be mentioned, I’m using “internet” as shorthand for the “deep” internet … not facebook. I’m talking websites that most people do not use, that you’d have to spend a lot of time on the internet to find. As such, the “internet” hypothesis would predict a greater bias towards smaller sub-reddits.
Anyway, I was mostly posing an alternate hypothesis. When I first noticed the trend on the personality forums, this is what I thought was happening -
Slacking off / internet addiction selects for Perceiving and low Conscientiousness.
Non-social-networking internet use selects for Introversion.
Any forum discussing an idea without immediate practical benefits selects for iNtuition.
And then, factor in lesswrong/giftedness...
If it’s a math/science/logic topic, it selects for Thinking and iNtuition.
High scores on Raven’s matrices select for Thinking, iNtuition. High scores on Working memory components select for Judging. The ACT/SAT additionally select for Conscientiousness
Strong mathematical affinity shifts those on the border of NTP and NTJ into *NTJ (people prefer dealing with intellectually ordered systems, even if they have messy rooms and chaotic lifestyles)
A scientific/engineering ideology creates a shift towards the concrete (empirical evidence, practical gains in technology, etc) shifts those on the border of NTJ and STJ into ISTJ.
In summary, I think LW and Mensa surveys are attracting a special subset of idea driven and logical people (iNtuitives and Thinkers) and likely to use the internet often/spot the survey. (Introverts)
I agree with (1), but would claim that it also selectively attracts introverts (and I’m unsure whether or not it will bias J-P to the P side)
(2) For each of these, I tried not to look at the data after finding the poll. I made predictions first. Just for fun / to correct for hindsight bias, anyone reading might want to do the same. To play, don’t click on the link or read my prediction until you make yours. Also, here is some data which claims to represent the general population—http://mbtitruths.blogspot.com/2011/02/real-statistics.html for comparison. I’ve already seen similar data on another site, so I won’t state my predictions on this one.
A website posts stats for people who have taken the test. Unlike the above simple random sample, this selects for internet users.
Prediction: I’d consider this “shallow internet”, so very weak biases to (I). The general population is (S), I’d expect a weak bias to (N) but not enough to overcome the general population’s S centering.
Result: apparently I suck at predictions. In hindsight all the top three would be predicted score high “Fi” on a Jungian cognitive function test, and Fi in theory would be more interested in taking personality tests. But that’s hindsight, and I’m not sure if connection between MBTI and Jung hasn’t been verified empirically.
Here is a “deep internet” forum that I wouldn’t ever visit… Christian singles chat forum! This should not suffer from the sample bias you mentioned earlier (He stated that websites I visit are likely to have users with similar personalities to me [ENTP])
Prediction: I tried my best not to look at the data despite the high visual salience as soon as you open that link. Here is my prediction: I’d predict strong biases towards Introversion (because internet), slight biases towards iNtuition (because religion is idea-based), moderate bias to Feeling (I think religious people are illogical) and … let’s say a slight tilt towards Judging. Call it a hunch, life experience says that Si (judging + sensing) is particularly predisposed to religion.
Result: OK, looks like my trends were right but my magnitude was way off. My “hunch” was correct but I didn’t listen to it closely enough and vastly underestimated the Judging bias, while my personal prejudice overestimated the Feeling bias. My predictions about intution and introversion were essentially correct though.
Prediction- Strong N, slight T bias. I don’t think T actually means “intelligent” as I define it, but I do think it would help on some portions of the IQ test.
Prediction—strong N, moderate T. I’m not sure about J-P. I think people who choose science tracks and go into academia will be P (creative types), whereas kids who get good grades but ultimately do not choose science will be J. I’m not sure which group they are looking at (I didn’t permit myself to read it yet, so I’m a bit vague on what exactly they did). I don’t think E—I will matter at all.
Result -
NT take high level science a lot more, Introverts take them slightly more. J-P is irrelevant.
Intuition really helps in school at all levels.
Feeling relates to high GPA in the easy courses but not the hard course (that’s pretty unexpected)
Introversion relates to high GPA in hard course but not in easy courses.
Percievers start out with a pretty big edge both in IQ and GPA in the lower level courses, Judging takes a slight lead in both those metrics in the advanced course. Not sure if this is noise.
Side finding—they also did IQ measurements. Again, only N related to IQ (in fact, F won out over T)...but it did not relate as much in the advanced courses. I think the advanced course chopped off the lower end of the IQ bell curve, leaving only smart Sensors. By the way, Extroverts have an IQ edge, despite getting lower grades and not taking advanced courses as often.
Thoughts? I think in general my ideas about introversion not mattering for intelligence, but mattering a lot for internet use, bear out. Apparently Thinking doesn’t really matter either...which I sort of felt was true, but I didn’t actually expect the IQ test scores to agree with me on that. It might have to do with self reported vs actual use of logic.
Of course, we are looking at the center of the bell curve, whereas on LW we are (presumably) looking at the far right edge.
They say that they found IQ correlates with I, N, T, and P. However, they claim that were surprised about the “I” correlation, because a large number of other studies have found that E is positively correlated. They go on to talk about how different testing conditions might favor E vs I. Some interesting further reading in there...it seems that N only correlates on the verbal reasoning section,
If you think about it in terms of Occam’s razor, what is the better explanation? That most people lied about their IQ, and fudged their SAT, ACT and personality type data to match, or that they’re telling the truth?
I’m inclined to believe the survey results myself, but there is a third possibility. If a certain personality type (or distribution of types) reflects a desire to associate with gifted people, or to be seen as gifted, we’d likely expect that to be heavily overrepresented in MENSA; that’s pretty much the reason the club exists, after all. We might also expect people with those desires to be less inclined to share average or poor IQ results, or even to falsify results.
If the same personality type is overrepresented here, then we have a plausible cause for similar personality test results and for exaggerated IQ reporting, without necessarily implying that the actual IQ distributions are similar.
I acknowledge that the sample set for the highest IQ groups are, of course, rather small, but that’s all we’ve got. What’s been happening with the numbers for the highest IQ groups, if indicative of what’s really happening, is not encouraging. The highest two groups have decreased in numbers while the lowest two have increased. Also, it looks like the prominence of each group has shifted over time such that the highest group went from being 1⁄5 to 1⁄20 and the moderately gifted and normal groups have grown substantially.
I don’t find it that hard to see why Lesswrong and Mensa would both select for introverted personalities. Do you?
I think most sensible people can deduce that IQ is positively correlated with SAT and ACT and all of them are positively correlated with “status”. I agree that SAT and ACT are more difficult to fudge though. I haven’t ever done either of them. Can they be easily redone several times? Do (smart) people liberally talk about their scores in the US?
Many people do IQ tests of different calibers several times and could just remember or report the best result they’ve gotten. There are different levels of dishonesty. “Lying” is a bit crude.
I don’t think anyone on Less Wrong has lied about their IQ. (addendum: not enough to seriously alter the results, anyway.) If you come up with a “valuing the truth” measure, LessWrong would score pretty highly on that considering the elaborate ways people who post here go about finding true statements in the first place. To lie about your IQ would mean you’d have to know to some degree what your real IQ is, and then exaggerate from there.
However, I do think it’s more likely than you mention that most people on LessWrong self-reporting IQ simply don’t know what their IQ is in absolutely certain terms, since to know your adult IQ you’d have to see a psychometricist and receive an administered IQ test. iqtesk.dk is normed by Mensa Denmark, so it’s far more reliable than self-reports. You don’t know where the self-reported IQ figures are coming from—they could be from a psychometricist measuring adult IQ, or they could be from somewhere far less reliable. It could be that they know their childhood IQ was measured at somewhere around 135 for example, and are going by memory. Or they could know by memory that their SAT is 99th percentile and spent a minute to look up what 99th percentile is for IQ, not knowing it’s not a reliable proxy. Or they might have taken an online test somewhere that gave ~140 and are recalling that number. Who knows? Either way, I consider “don’t attribute to malice what you can attribute to cognitive imperfection” a good mantra here.
126 is actually higher than a lot of people think. As an average for a community, that’s really high—probably higher than all groups I can think of except math professors, physics professors and psychometricists themselves. It’s certainly higher than the averages for MIT and Harvard, anyway.
About the similarity between self-reported IQ and SAT scores: SAT scores pre-1994 (which many of the scores on here are not likely to fall into) are not reliable as IQ test proxies; Mensa no longer accepts them. This is because it is much easier to game. I tutor the SAT, and when I took the SAT prior to applying at a tutor company my reading score was 800, but in high school pre-college it was only in the mid-600s. SAT scores in reading are heavily influenced by (1) your implicit understanding of informal logic, and (2) your familiarity with English composition and how arguments/passages may be structured. Considering the SAT has contained these kinds of questions since the mid-90s, I am inclined to throw its value as a proxy IQ test out the window and don’t think you can draw conclusions about LessWrong’s real collective IQ from the reported SAT scores.
The IQTest.dk result may have given the lowest measure, but I also think it’s the most accurate measure. It would not put LessWrong in the 130s, maybe, but it would mean that the community is on the same level of intellect as, say, surgeons and Harvard professors, which is pretty formidable for a community.
I don’t think anyone on Less Wrong has lied about their IQ.
Over 1000 people took the test. Statistically speaking, it should have included about 50 sociopaths. Not all sociopaths would necessarily lie on that question but considering that you’re going to have to explain why you think that none of the sociopaths lied (or pathological liars or borderlines or other types that are likely to have been included in the test results) you have chosen a position, or at least wording, which is going to be darned near impossible for you to defend.
To lie about your IQ would mean you’d have to know to some degree what your real IQ is, and then exaggerate from there.
No because to say “I know my IQ” when one doesn’t is also a lie, and that’s what it would be saying if they put ANY IQ in the box without knowing it.
iqtesk.dk is normed by Mensa Denmark, so it’s far more reliable than self-reports.
Mensa is a club not a professional IQ testing center. They’re not even legally allowed to give out scores anymore. Their test scores are not considered to be accurate. For one thing, they (and iqtest.dk) do not evaluate for learning disorders. One in six gifted people has a learning disorder. Learning disorders lower one’s score and so the test should be adjusted to reflect this.
The iqtest.dk scores ARE self-reported. That is to say, the user types the IQ score into the survey box themselves. In that way, they’re equally flawed to the other intelligence questions, not “more reliable than self-reports”.
I stopped here because the rest of the comment follows the same pattern. About every other sentence in your comment is irrational. LessWrong is going to eat you alive, honey. Get out while you’re ahead.
Over 1000 people took the test. Statistically speaking, it should have included about 50 sociopaths.
Not if LessWrong values truthseeking activities more than the general population, or considers lying/truth-fabrication a greater sin than the general population does, or if LessWrong just generally attracts less sociopaths than the general population. If over 1000 fitness enthusiasts take a test about weight, the statistics re: obesity are not going to reflect the general population’s. Considering the CRT scores of LessWrong and the nature of this website to admire introspection and truthseeking activities, I doubt that LW would be reflective of the general population in this way.
Lies are more than untrue statements; at least, in the context of self-reports, they are conscious manipulations of what one knows to be true. Someone might think they know their IQ because they’ve taken less reliable IQ tests, or because they had a high childhood IQ, or because they extrapolated their IQ from SAT scores, or for a host of other reasons. In this case they haven’t actually lied, they’ve just stated something inaccurate.
Someone could put an IQ when they have no idea what their IQ is, yes, in the sense that they have never taken a test of any sort and have no idea what their IQ would be if they took one, even an inaccurate one. I don’t think many people here would do that, though, because of the truthseeking reasons mentioned earlier.
Mensa is a club not a professional IQ testing center. They’re not even legally allowed to give out scores anymore. Their test scores are not considered to be accurate.
Mensa doesn’t need to be a professional IQ testing center for their normings to be accurate, however. I am also not sure how not accounting for learning disorders would seriously alter IQTest.dk’s validity over self-reports.
However, it’s inaccurate to say that because someone puts their number in the box from IQTest.dk that they’re “equally flawed” to the other intelligence questions. Someone who self-reports an IQ number, any number, may not know if that number was obtained using accurate methodology. It may be an old score from childhood, and childhood IQ scores vary wildly compared to adult IQ scores. It may be an extrapolation from SAT scores, as I mentioned above. There are a number of ways in which self-reported IQ differs from reported IQtest.dk IQ.
LessWrong is going to eat you alive, honey. Get out while you’re ahead.
This reads as unnecessarily tribalistic to me. I take it you think I am an undiscriminating skeptic? In any case, cool it.
I’d expect Less Wrongers to be more likely to be sociopaths than average. We’re generally mentally unusual.
Yeah, I am perfectly aware that the IQ score I got when I was three wasn’t valid then and certainly isn’t now. The survey didn’t ask “What’s a reasonable estimate of your IQ?”.
Not if LessWrong values truthseeking activities more than the general population, or considers lying/truth-fabrication a greater sin than the general population does, or if LessWrong attracts less sociopaths than the general population. If over 1000 fitness enthusiasts take a test about weight, the statistics re: obesity are not going to reflect the general population’s. Considering the CRT scores of LessWrong and the nature of this website to admire introspection and truthseeking activities, I doubt that LW would be reflective of the general population in this way.
That is why I used the wording “statistically speaking”—it is understood to mean that I am working from statistics that were generated on the overall population as opposed to the specific population in question. You are completely ignoring my point which is that you have chosen a position which is going to be more or less impossible to defend. That position was:
I don’t think anyone on Less Wrong has lied about their IQ.
It’s considered very rude to completely ignore someone’s argument and nit pick at their wording. That is what you just did.
Lies are more than untrue statements; at least, in the context of self-reports,
Now it’s like you’re trying to make up a new definition of the word lying so you can continue to think your ridiculous assessment that:
To lie about your IQ would mean you’d have to know to some degree what your real IQ is
By the common definition of the word “lie” producing a number when you do not know the number definitely does qualify as a lie. You’re not fooling me by trying to make a new definition of the word “lie” in this context. This behavior just looks ridiculous to me.
Mensa doesn’t need to be a professional IQ testing center for their normings to be accurate, however.
But they do need to provide a professional IQ testing service if they want their norms to mean something. The iqtest.dk might turn out to be a better indicator of visual-spatial ability than IQ, or it might discriminate against autistics, which LW might have an unusually large number of (seeing as how there are a lot of CS people here).
However, it’s inaccurate to say that because someone puts their number in the box from IQTest.dk that they’re “equally flawed” to the other intelligence questions.
Here you go twisting my wording. I specifically said:
In that way, they’re equally flawed to the other intelligence questions...
The only reason I’m responding to you is because I am hoping you will see that you need to do more work on your rationality. Please consider getting some rationality training or something.
The general population would contain 50 sociopaths to 1000; I don’t think LessWrong contains 50 sociopaths to 1000. Rationality is a truth-seeking activity at its core, and I suspect a community of rationalists would do their best to avoid lying consciously.
I am not sure what “the common definition of the word ‘lie’” is, especially since there are a lot of differing interpretations of what it means to lie. I know that wrong answers are distinct from lies, however. I think that a lot of LessWrong people might have put an IQ that does not reflect an accurate result. But I doubt that many LessWrong people have put a deliberately inaccurate result for IQ. Barring “the common definition” (I don’t know what that is), I’m defining “stating something when you know what you are stating is false” as a lie, since someone can put a number when they don’t know for sure what the true number is but don’t know that the number they are stating is false either.
I don’t know what you mean by “mean something” with respect to Mensa Denmark’s normings. They will probably be less accurate than a professional IQ testing service, but I don’t know why they would be inaccurate or “meaningless” by virtue of their organization not being a professional IQ testing service.
The only way I can think of in which the self-reported numbers would be more accurate than the IQTest.dk numbers is if the LW respondents knew that their IQ numbers were from a professional testing service and they had gone to this service recently. But since the test didn’t specify how they obtained this self-report, I can’t say, nor do I think it’s likely.
IQTest.dk uses Raven’s Progressive Matrices which is a standard way to measure IQ across cultures. This is because IQ splits between verbal/spatial are not as common. It wouldn’t discriminate against autistics, because it actually discriminates in favor of autistics; people with disorders on the autism spectrum are likely to score higher, not lower.
I’m not sure how the bolding of “in that way” bolsters your argument. Paraphrased, it would be “in the way that the user types the IQ score into the survey box themselves, the IQTest.dk questions are equally flawed to the other intelligence questions.” But this neglects to consider that the source of the number is different; they are self-reports in the sense that the number is up to someone to recall, but if someone types in their IQTest.dk number you know it came from IQTest.dk. If someone types in their IQ without specifying the source, you have no idea where they got that number from—they could be estimating, it could be a childhood test score, and so on.
Please consider getting some rationality training or something.
Remarks like these are unnecessary, especially since I’ve just joined the site.
Over 1000 people took the test. Statistically speaking, it should have included about 50 sociopaths.
Do you have statistics about how many sociopaths take extra-long online tests or how many sociopaths frequent rationalist forums? Or are you just talking about the percentage of sociopaths in the general population?
As a sidenote one would think that people willing to lie about their IQ would be positively correlated with people that look up Bayes’ birthdate before filling in their ‘estimation’. Anyone making a statistical analysis regarding this?
LessWrong is going to eat you alive, honey.
Downvoted for this statement and overall unnecessary rude tone.
I agree with that article and that’s exactly why I downvoted you. You were contemptuously calling someone ‘honey’ and behaving like an all-around dick that smirks at newcomers and warns them that the rest of us will chew them up. That’s not what I want to see belong here and I won’t be a pacifist about it when you’re behaving like a weed.
I agree with that article and that’s exactly why I downvoted you.
I read the article again very carefully, trying to figure out whether Eliezer was advocating weeding people for behaving the way that I did. The article is about keeping “fools” out of the “garden of intelligent discussion”. It says nothing about the tone of posts and what tones should be weeded.
contemptuously calling someone ‘honey’
Actually, that was intended to make the tone friendlier. I acknowledge that this is not the way that you perceived it. My feeling is not contempt. I just don’t think he is likely to contribute constructively.
behaving like an all-around dick that smirks at newcomers and warns them that the rest of us will chew them up
I like newcomers. The sole problem here was that about half of what this specific newcomer said was irrational.
I won’t be a pacifist about it when you’re behaving like a weed.
That is exactly how I felt when I saw alfredmacdonald posting a bunch of irrational thoughts in this place for rationality. Are we both doing something wrong, then? The tone of your last comment doesn’t look any different to me than the tone of my comments. Is it that you feel that using a tone like this is never justified and we’ve both made a mistake or is it that you believe it’s okay to speak like this to people you feel are rude, but not to people you think are being irrational?
Is it that you feel that using a tone like this is never justified and we’ve both made a mistake or is it that you believe it’s okay to speak like this to people you feel are rude
I speak to you like this because the simple explanation of “I downvoted you for excessive rudeness” doesn’t seem to satisfy you, to the point that you keep asking me for further clarification (and re-asked me when I ignored your first question). So I have to change my tone, because though the repetition of the same clarification “I downvoted you for excessive rudeness” should be adequate, you don’t get it.
Let me mention that I won’t continue discussing this, and if you continue pestering me you’ll be incentivize me to not offer any clarification at all for future downvotes towards your person, to just downvote you without explanation.
I see that you’re not interested in discussing the original issue that started this. I know that everyone has limited energy, so I accept this. It feels important to mention that none of my comments were written with an intent to pester you. I am not disagreeing with you about how you experienced them—different people experience things differently. I only mean to tell you that I did not intend to cause you this experience.
My intent was to understand your point of view better to see if our disagreement over whether a cold tone is justified for the purpose of garden-keeping would be resolved or if I would learn anything.
I hope you can see that despite our disagreement about how to protect the quality of the discussion area, we both care about whether the quality of the discussion area is good, and are willing to take action to protect it. I am not trying to troll; this wasn’t for “lulz”. I am doing it because I care. We have that one thing in common.
For this reason, I would prefer to use a friendly or neutral tone with you in the future. You may or may not be interested in putting this difference aside in order to have smoother interactions in the future, but I am willing to, so I invite you to do the same.
Do you have statistics about how many sociopaths take extra-long online tests …
The verbiage “statistically speaking” was supposed to imply an acknowledgement that I know that the statistics were based on the overall population, not the specific context.
As a sidenote one would think that people willing to lie about their IQ would be positively correlated with people that look up Bayes’ birthdate before filling in their ‘estimation’. Anyone making a statistical analysis regarding this?
Ooh. This is a very, very good point. And if the survey participants really wanted to look gifted, they’d have probably decided that fudging the Bayes question was a necessity. I added your thought to my IQ accuracy comment. Upvote.
Downvoted for this statement and overall unnecessary rude tone.
Thank you for explaining your downvote.
Is it that you disagree with Well-Kept Gardens Die By Pacifism or that you do this in a different way? If you have some different method, what is it?
Note: It’s probably inevitable that someone will ask me why I seem to agree with the spirit of this article, if I don’t believe in “elitism”. My answer is, succinctly, that humans seek like-minded people to hang out with, that this is part of fulfilling one’s social needs, and that it’s silly to allow our attempt to get basic needs met to be politicized and called “elitism” just because we gather around intellectualism when it is no different from the desire of a single mom to spend some time out with adults because children can’t have the same conversations or the desire of hunting-minded people to engage in activities without vegetarians harping on them (or vice versa).
On IQ Accuracy:
As Yvain says, “people have been pretty quick to ridicule this survey’s intelligence numbers as completely useless and impossible and so on” because if they’re true, it means that the average LessWronger is gifted. Yvain added a few questions to the 2012 survey, including the ACT and SAT questions and the Myers-Briggs personality type question that I requested (I’ll explain why this is interesting), and that give us a few other things to check against, which has made the figures more believable. The ridicule may be an example of the “virtuous doubt” that Luke warns about in Overconfident Pessimism, so it makes sense to “consider the opposite”:
The distribution of Myers-Briggs personality types on LessWrong replicates the Mensa pattern. This is remarkable since the patterns of personality types here are, in many significant ways, the exact opposite of what you’d find in the regular population. For instance, the introverted rationalists and idealists are each about 1% of the population. Here, they are the majority and it’s the artisans and guardians who are relegated to 1% or less of our population.
Mensa’s personality test results were published in the December 1993 Mensa Bulletin. Their numbers.
So, if you believe that most of the people who took the survey lied about their IQ, you also need to believe all of the following:
That most of these people also realized they needed to do IQ correlation research and fudge their SAT and ACT scores in order for their IQ lie to be believable.
Some explanation as to why the average of lurker’s IQ scores would come out so close to the average of poster’s IQ scores. The lurkers don’t have karma to show off, and there’s no known incentive good enough to get so many lurkers to lie about their IQ score. Vaniver’s figures.
Some explanation for why the personality type pattern at LessWrong is radically different from the norm and yet very similar to the personality type pattern Mensa published and also matched my predictions. Even if they had knowledge of the Mensa personality test results and decided to fudge their personality type responses, too, they somehow managed to fudge them in such a way that their personality types accidentally matched my predictions.
That they decided not to cheat when answering the Bayes birthday question even though they were dishonest enough to lie on the IQ question, motivated to look intelligent, and it takes a lot less effort to fudge the Bayes question than the intelligence and personality questions. (This was suggested by ArisKatsaris).
That both posters and lurkers had some motive strong enough to justify spending 20+ minutes doing the IQ correlation research and fudging personality test questions while probably bored of ticking options after filling out most of a very long survey.
It’s easier just to put the real number in the IQ box than do all that work to make it believable, and it’s not like the liars are likely to get anything out of boasting anonymously, so the cost-benefit ratio is just not working in favor of the liar explanation.
If you think about it in terms of Occam’s razor, what is the better explanation? That most people lied about their IQ, and fudged their SAT, ACT and personality type data to match, or that they’re telling the truth?
Summary of criticism:
Possible Motive to Lie: The desire to be associated with a “gifted” group:
In re to this post, it was argued by NonComposMentis that a potential motive to lie is that if the outside world perceives LessWrong as gifted, then anyone having an account on LessWrong will look high-status. In rebuttal:
I figure that lurkers would not be motivated to fudge their results because they don’t have a bunch of karma on their account to show off and anybody can claim to read LessWrong, so fudging your IQ just to claim that the site you read is full of gifted people isn’t likely to be motivating. I suggested that we compare the average IQs of lurkers and others. Vaniver did the math and they are very, very close..
I argued, among other things, that it would be falling for a Pascal’s mugging to believe that investing the extra time (probably at least $5 worth of time for most of us) into fudging the various different survey questions is likely to contribute to a secret conspiracy to inflate LessWrong’s average IQ.
Did the majority avoid filling out intelligence related questions, letting the gifted skew the results?
Short answer: 74% of people answered at least one intelligence related question and since most people filled out only one or two, the fact that the self-report, ACT and SAT score averages are so similar is remarkable.
I realized, while reading Vaniver’s post that if only 1⁄3 of the survey participants filled out the IQ score, this may have been due to something which could have skewed the results toward the gifted range, for instance, if more gifted people had been given IQ tests for schooling placement (and the others didn’t post their IQ score because they did not know it) or if the amount of pride one has in their IQ score has a significant influence on whether one reported it.
So I went through the data and realized that most of the people who filled out the IQ test question did not fill out all the others. That means that 804 people (74% not 33%) answered at least one intelligence related question. As we have seen, the IQ correlations for the IQ, SAT and ACT questions were very close to each other (unsurprisingly, it looks like something’s up with the internet test… removing those, it’s 63% of survey participants that answered an intelligence related question). It’s remarkable in and of itself that each category of test scores generated an average IQ so similar to the others considering that different people filled them out. I mean if 1⁄3 of the population filled out all of the questions, and the other 2⁄3 filled out none, we could say “maybe the 1⁄3 did IQ correlation research and fudged these” but if most of the population fills out one or two, and the averages for each category come out close to the averages for the other categories, why is that? How would that happen if they were fudging?
It does look to me like people gave whatever test scores they had and that not all the people had test scores to give but it does not look to me like a greater proportion of the gifted people provided an intelligence related survey answer. Instead it looks like most people provided an intelligence related survey answer and the average LessWronger is gifted.
Exploration of personality test fudging:
Erratio and I explored how likely it is that people could successfully fudge their personality tests and why they might do that.
There are a lot of questions on the personality test that have an obvious intelligence component, so it’s possible that people chose the answer they thought was most intelligent.
There are also intelligence related questions where it’s not clear which answer is most intelligent. I listed those.
The intelligence questions would mostly influence the sensing/intuition dichotomy and the thinking/feeling dichotomy. This does not explain why the extraversion/introversion and perceiving/judging results were similar to Mensa’s.
Alternate possibility: The distribution of personality types in Mensa/LW relative to everyone else is an artifact produced by self-identified smart people trying to signal their intelligence by answering ‘yes’ to traits that sound like the traits they ought to have.
eg. I know that a number of the T/F questions are along the lines of “I use logic to make decisions (Y/N)”, which is a no-brainer if you’re trying to signal intelligence.
A hypothetical way to get around this would be to have your partner/family member/best friend next to you as you take the test, ready to call you out when your self-assessment diverges from your actual behaviour (“hold on, what about that time you decided not to go to the concert of [band you love] because you were angry about an unrelated thing?”)
Ok, it’s possible that all of the following happened:
Most of the 1000 people decided to lie about their IQ on the LessWrong survey.
Most of the liars realized that their personality test results were going to be compared with Mensa’s personality type results, and it dawned on them that this would bring their IQ lie into question.
Most of the liars decided that instead of simply skipping the personality test question, or taking it to experience the enjoyment of finding out their type, they were going to fudge the personality test results, too.
Most of the liars actually had the patience to do an additional 72 questions specifically for the purpose of continuing to support a lie when they had just slogged through 100 questions.
Most of the liars did all of that extra work (Researching the IQ correlation with the SAT and the ACT and fudging 72 personality type questions) when it would have been so much easier to put their real IQ in the box, or simply skip the IQ question completely because it is not required.
Most of the liars succeeded in fudging their personality types. This is, of course, possible, but it it is likely to be more complicated than it at first seems. They’d have to be lucky that enough of the questions give away their intelligence correlation in the wording (we haven’t verified that). They’d have to have enough of an understanding of what intelligent people are like that they’d choose the right ones. Questions like these are likely to confuse a non-gifted person trying to guess which answers will make them look gifted:
“You are more interested in a general idea than in the details of its realization”
(Do intelligent people like ideas or details more?)
“Strict observance of the established rules is likely to prevent a good outcome”
(Either could be the smarter answer, depending who you ask.)
“You believe the best decision is one that can be easily changed”
(It’s smart to leave your options open, but it’s also more intellectually self-confident and potentially more rewarding to take a risk based on your decision-making abilities.)
“The process of searching for a solution is more important to you than the solution itself”
(Maybe intelligence makes playing with ideas so enjoyable, gifted people see having the solution as less important.)
“When considering a situation you pay more attention to the current situation and less to a possible sequence of events”
(There are those that would consider either one of these to be the smarter one.)
There were a lot of questions that you could guess are correlated with intelligence on the test, and some of them are no-brainers, but are there enough of those no-brainers with obvious intelligence correlation that a non-gifted person intent on looking as intelligent as possible would be able to successfully fudge their personality type?
The massive fudging didn’t create some totally unexpected personality type pattern. For instance, most people are extraverted. Would they realize the intelligence implications and fudge enough extravert questions to replicate Mensa’s introverted pattern? Would they know to choose the judging questions over the perceiving questions would make them look like Mensans? It makes sense that the thinking vs. feeling and intuiting vs sensing metrics would use questions that would be of the type you’d obviously need to fudge, but why would they also choose introvert and judging answers?
The survey is anonymous and we don’t even know which people gave which IQ responses, let alone are they likely to receive any sort of reward from fudging their IQ score. Can you explain to me:
What reward would most of LessWrong want to get out of lying about their IQs?
Why, in an anonymous context where they can’t even take credit for claiming the IQ score they provided, most of LessWrong is expecting to receive any reward at all?
Can you explain to me why fudged personality type data would match my predictions? Even if they were trying to match them, how would they manage it?
“Lie” is a strawman. One could report an estimate, mis-remember, report the other “IQ” (mental age / chronological age metric), or one may have took any one of entirely faulty online tests that report IQ as high to increase the referral rate (some are bad enough to produce >100 if the answers are filled in at random).
This would be a good point in the event that we were not discussing IQ scores generated by an IQ test selected by Yvain, which many people took at the same time as filling out the survey. This method (and timing) rules out problems due to relying on estimates alone, most of the potential for mis-remembering, (neither of which should be assumed to be likely to result in an average score that’s 30 points too high, as mistakes like these could go in either direction), and, assuming that the IQ test Yvain selected was pretty good, it also rules out the problem of the test being seriously skewed. If you would like to continue this line of argument, one effective method of producing doubt would be to go to the specific IQ test in question, fill out all of the answers randomly, and report the IQ that it produces. If you want to generate a full-on update regarding those particular test results, complete with Yvain being likely to refrain from recommending this source during his next survey, write a script that fills out the test randomly and reports the results so that multiple people can run it and see for themselves what average IQ the test produces after a large number of trials. You may want to check to see whether Yvain or Gwern or someone has already done this before going to the trouble.
Also, there really were people whose concern it was that people were lying on the survey. Your “lie is a strawman” perception appears to have been formed due to not having read the (admittedly massive number of) comments on this.
Look. People misremember (and remember the largest value, and so on) in the way most favourable to themselves. While mistakes can of course go in either direction, they don’t actually go in either direction. If you ask men to report their penis size (quite literally), they over-estimate; if you ask them to measure, they still overestimate but not by as much. This sort of error is absolutely the norm in any surveys. More so here, as the calibration (on Bayes date of birth question at least) was comparatively very bad.
The situation is anything but symmetric, given that the results are rather far from the mean, on a Gaussian.
Furthermore, given the interest in self improvement, people here are likely to have tried to improve their test scores by practice, which would have considerably lower effect on iqtest.dk unless you practice specifically the Raven’s matrices.
The low scores on iqtest.dk are particularly interesting in light that the scores on the latter are a result of better assignment of priors / processing of probabilities (as, fundamentally, one needs to pick the choice which results in simplest—highest probability—overall pattern. If one is overconfident about the pattern they see being the best, one’s score is lowered, so poor calibration will hurt that test more).
I intuit that this is likely to be a popular view among sceptics, but I do not recall ever being presented with research that supports this by anyone. To avoid the lure of “undiscriminating scepticism”, I am requesting to see the evidence of this.
I agree that, for numerous reasons, self-reported IQ scores, SAT scores, ACT scores and any other scores are likely to have some amount of error, and I think it’s likely for the room for error to be pretty big. On that we agree.
An average thirty points higher than normal seems to me to be quite a lot more than “pretty big”. That’s the difference between an IQ in the normal range and an IQ large enough to qualify for every definition of gifted. To use your metaphor, that’s like having a 6-incher and saying it’s 12. I can see guys unconsciously saying it’s 7 if it’s 6, or maybe even 8. But I have a hard time believing that most of these people have let their imaginations run so far away with them as to accidentally believe that they’re Mensa level gifted when they’re average. I’d bet that there was a significant amount of error, but not an average of 30 points.
If you agree with those two, then whether we agree over all just depends on what specific belief we’re each supporting.
I think these beliefs are supported:
The SAT, ACT, self-reported IQ and / or iqtest.dk scores found on the survey are not likely to be highly accurate.
Despite inaccuracies, it’s very likely that the average LessWrong member has an IQ above average—in other words, I don’t think that the scores reported on the survey are so inaccurate that I should believe that most LessWrongers actually have just an average IQ.
LessWrong is (considering a variety of pieces of evidence, not just the survey) likely to have more gifted people than you’d find by random chance.
Do we agree on those three beliefs?
If not, then please phrase the belief(s) you want to support.
Even if every self-reported IQ is exactly correct, the average of the self-reported IQ values can still be (and likely will still be) higher than the average of the readership’s IQ values.
Consider two readers, Tom and Jim. Tom does an IQ test, and gets a result of 110. Jim does an IQ test, and gets a result of 90. Tom and Jim are both given the option to fill in a survey, which asks (among other questions) what their IQ is. Neither Tom nor Jim intend to lie.
However, Jim seems significantly more likely to decide not to participate; while Tom may decide to fill in the survey as a minor sort of showing off. This effect will skew the average upwards. Perhaps not 30 points upwards… but it’s an additional source of bias, independent of any bias in individual reported values.
I remember looking into this when I looked at the survey data. There were only a handful of people who reported two-digit IQs, which is consistent with both the concealment hypothesis and the high average intelligence hypothesis. If you assume that nonresponders have an IQ of 100 on average the average IQ across everyone drops down to 112. (I think this is assumption is mostly useful for demonstrative purposes; I suspect that the prevalence of people with two-digit IQs on LW is lower than in the general population.)
(You could do some more complicated stuff if you had a functional form for concealment that you wanted to predict, but it’s not obvious to me that IQs on LW actually follow a normal distribution, which would make it hard to separate out the oddities of concealment with the oddities of the LW population.)
Ah! Good point! Karma for you! Now I will think about whether there is a way to figure out the truth despite this.
Ideas?
Hmmm. Tricky.
Select a random sampling of people (such as by picking names from the phonebook). Ask each person whether they would like to fill in a survey which asks, among other things, for their IQ. If a sufficiently large, representative sample is taken, the average IQ of the sample is likely to be 100 (confirm if possible). Compare this to the average reported IQ, in order to get an idea of the size of the bias.
Select a random sampling of lesswrongers, and ask them for their IQs. If they all respond, this should cut out the self-selection bias (though the odds are that at least some of them won’t respond, putting us back at square one).
It’s probably also worth noting that this is a known problem in statistics which is not easy to compensate for.
There’s also the selection effect of only getting answers from “people who , when asked, can actually name their IQ”.
As one of the sceptics, I might as well mention a specific feature of the self-reported IQs that made me pretty sure they’re inflated. (Even before I noticed this feature, I expected the IQs to be inflated because, well, they’re self-reported. Note that I’m not saying people must be consciously lying, though I wouldn’t rule it out. Also, I agree with your three bullet points but still find an average LW IQ of 138-139 implausibly high.)
The survey has data on education level as well as IQ. Education level correlates well with IQ, so if the self-reported IQ & education data are accurate, the subsample of LWers who reported having a “high school” level of education (or less) should have a much lower average IQ. But in fact the mean IQ of the 34% of LWers with a high school education or less was 136.5, only 2.2 points less than the overall mean.
There is a pretty obvious bias in that calculation: a lot of LWers are young and haven’t had time to complete their education, however high their IQs. This stacks the deck in my favour because it means the high-school-or-less group includes a lot of people who are going to get degrees but haven’t yet, which could exaggerate the IQ of the high-school-or-less group.
I can account for this bias by looking only at the people who said they were ≥29 years old. Among that older group, only 13% had a high school education or less...but the mean IQ of that 13% was even higher* at 139.6, almost equal to the mean IQ of 140.0 for older LWers in general. The sample sizes aren’t huge but I think they’re too big to explain this near-equality away as statistical noise. So IQ or education level or age was systematically misreported, and the most likely candidate is IQ, ’cause almost everyone knows their age & education level, and nerds probably have more incentive to lie on a survey about their IQ than about their age or education level.
* Assuming people start university at age 18, take 3 years to get a bachelor’s, a year to get a master’s, and then up to 7 years to get a PhD, everyone who’s going to get a PhD will have one at age 29. In reality there’re a few laggards but not enough to make much difference; basically the same result comes out if I use age 30 or age 35 as a cutoff.
And I suspect if you look at the American population for that age cohort, you’ll find a lot higher a percentage than 13% which have a “high school education or less”… All you’ve shown is that of the highschool-educated populace, LW attracts the most intelligent end, the people who are the dropouts for whatever reason. Which for high-IQ people is not that uncommon (and one reason the generic education/IQ correlation isn’t close to unity). LW filters for IQ and so only smart highschool dropouts bother to hang out here? Hardly a daring or special pleading sort of suggestion. And if we take your reasoning at face-value that the general population-wide IQ/education correlate must hold here, it would suggest that there would be hardly any autodidacts on LW (clearly not the case), such as our leading ‘high school education or less’ member, Eliezer Yudkowsky.
Right, but even among LWers I’d still expect the dropouts to have a lower average IQ if all that’s going on here is selection by IQ. Sketch the diagram. Put down an x-axis (representing education) and a y-axis (IQ). Put a big slanted ellipse over the x-axis to represent everyone aged 29+.
Now (crudely, granted) model the selection by IQ by cutting horizontally through the ellipse somewhere above its centroid. Then split the sample that’s above the horizontal line by drawing a vertical line. That’s the boundary between the high-school-or-less group and everyone else. Forget about everyone below the horizontal line because they’re winnowed out. That leaves group A (the high-IQ people with less education) and group B (the high-IQ people with more).
Even with the filtering, group A is visibly going to have a lower average IQ than B. So even though A comprises “the most intelligent end” of the less educated group, there remains a lingering correlation between education level and IQ in the high-IQ sample; A scores less than B. The correlation won’t be as strong as the general population-wide correlation you refer to, but an attenuated correlation is still a correlation.
It seems implausible to me that education level would screen off the same parts of the IQ distribution in LW as it does in the general population, at least at its lower levels. It’s not too unreasonable to expect LWers with PhDs to have higher IQs than the local mean, but anyone dropping out of high school or declining to enter college because they dislike intellectual pursuits, say, seems quite unlikely to appreciate what we tend to talk about here.
Upvoted. If I repeat the exercise for the PhD holders, I find they have a mean IQ of 146.5 in the older subsample, compared to 140.0 for the whole older subsample, which is consistent with what you wrote.
How significant is that difference?
I did a back-of-the-R-session guesstimate before I posted and got a two-tailed p-value of roughly 0.1, so not significant by the usual standard, but I figured that was suggestive enough.
Doing it properly, I should really compare the PhD holders’ IQ to the IQ of the non-PhD holders (so the samples are disjoint). Of the survey responses that reported an IQ score and an age of 29+, 13 were from people with PhDs (mean IQ 146.5, SD 14.8) and 135 were from people without (mean IQ 139.3, SD 14.3). Doing a t-test I get t = 1.68 with 14.2 degrees of freedom, giving p = 0.115.
It’s a third of a SD and change (assuming a 15-point SD, which is the modern standard), which isn’t too shabby; comparable, for example, with the IQ difference between managerial and professional workers. Much smaller than the difference between the general population and PhDs within it, though; that’s around 25 points.
I was really asking about sample size, as I was too lazy to grab the raw data.
Yes, and even without particular expectation of inflation, once you see IQs that are very high, you can be quite sure IQs tend to be inflated simply because of the prior being the bell curve.
Any time I see “undiscriminating scepticism” mentioned, it’s a plea to simply ignore necessarily low priors when evidence is too weak to change conclusions. Of course, it’s not true “undiscriminating scepticism”. If LW undergone psychologist-administered IQ testing and that were the results, and then there was a lot of scepticism, you could claim that there’s some excessive scepticism. But as it is, rational processing of probabilities is not going to discriminate that much based on self reported data.
Sceptics in that case, I suppose, being anyone who actually does the most basic “Bayesian” reasoning, such as starting with a Gaussian prior when you should (and understanding how an imperfect correlation between self reported IQ and actual IQ would work on that prior, i.e. regression towards the mean when you are measuring by proxy). I picture there’s a certain level of Dunning Kruger effect at play, whereby those least capable of probabilistic reasoning would think themselves most capable (further evidenced by calibration; even though the question may have been to blame, I’m pretty sure most people believed that a bad question couldn’t have that much of an impact).
Wikipedia to the rescue, given that a lot of stuff is behind the paywall...
http://en.wikipedia.org/wiki/Illusory_superiority#IQ
“The disparity between actual IQ and perceived IQ has also been noted between genders by British psychologist Adrian Furnham, in whose work there was a suggestion that, on average, men are more likely to overestimate their intelligence by 5 points, while women are more likely to underestimate their IQ by a similar margin.”
and more amusingly
http://en.wikipedia.org/wiki/Human_penis_size#Erect_length
Just about any internet forum would select for people owning a computer and having an internet connection and thus cut off the poor, mentally disabled, and so on, improving the average. So when you state it this way—mere “above average”—it is a set of completely unremarkable beliefs.
It’d be interesting to check how common are advanced degrees among white Americans with actual IQ of 138 and above, but I can’t find any info.
This was one of the things I checked when I looked into the IQ results from the survey here and here. One of the things I thought was particularly interesting was that there was a positive correlation between self-reported IQ and iqtest.dk (which is still self-reported, and could have been lied on, but hopefully this is only deliberate lies, rather than fuzzy memory effects) among posters and a negative correlation among lurkers. This comment might also be interesting.
I endorse Epiphany’s three potential explanations, and would quantify the last one: I strongly suspect the average IQ of LWers is at least one standard deviation above the norm. I would be skeptical of the claim that it’s two standard deviations above the norm, given the data we have.
Wow, that’s quite interesting—that’s some serious Dunning-Kruger. Scatterplot could be of interest.
Thing to keep in mind is that even given a prior that errors can go either way equally, when you have obtained a result far from the mean, you must expect that errors (including systematic errors) were predominantly in that direction.
Other issue is that in a 1000 people, about 1 will have an IQ of >=146 or so , while something around 10 will have fairly severe narcissism (and this is not just your garden variety of overestimating oneself, but the level where it interferes with normal functioning).
Self reported IQ of 146 is thus not really a good sign overall. Interestingly some people do not understand that and go on how the others “punish” them for making poorly supported statements of exceptionality, while it is merely a matter of correct probabilistic reasoning.
The actual data is even worse than what comparisons of prevalence would suggest − 25% of people put themselves in the top 1% in some circumstances.
Yes, average of 115 would be possible.
The actual data is linked in the post near the end. If you drop three of the lurkers- who self-reported 180, 162, and 156 but scored 102, 108, and 107- then the correlation is positive (but small). (Both samples look like trapezoids, which is kind of interesting, but might be explained by people using different standard deviations.)
That sounds pretty high to me. I haven’t looked into narcissism as such, but I remember seeing similar numbers for antisocial personality disorder when I was looking into that, which surprised me; the confusion went away, however, when I noticed that I was looking at the prevalence in therapy rather than the general population.
Something similar, perhaps?
You know, people do lie to themselves. It’s a sad but true (and well known around here) fact about human psychology that humans have surprisingly bad models of themselves. It is simply true that if you asked a bunch of people selected at random about their (self-reported) IQ scores, you would get an average of more than 100. One would hope that LessWrongers are good enough at detecting bias in order to mostly dodge that bullet, but the evidence of whether or not we actually are that good at it is scarce at best.
Your unintentional lie explanation does not explain how the SAT scores ended up so closely synchronised to the IQ scores—as we know, one common sign of a lie is that the details do not add up. Synchronising one’s SAT scores to the same level as one’s IQ scores would most likely require conscious effort, making the discrepancy obvious to the LessWrong members who took the survey. If you would argue that they were likely to have chosen corresponding SAT scores in some way that did not require them to become consciously aware of discrepancies in order to synchronize the scores, how would you support the argument that they synched them on accident? If not, then would you support the argument that LessWrong members consciously lied about it?
Linda Silverman, a giftedness researcher, has observed that parents are actually pretty decent at assessing their child’s intellectual abilities despite the obvious cause for bias.
“In this study, 84% of the children whose parents indicated that they fit three-fourths of the characteristics tested above 120 IQ. ” (An unpublished study, unfortunately.)
http://www.gifteddevelopment.com/PDF_files/scalersrch.pdf
This isn’t exactly the same as managing knowledge of one’s own intellectual abilities, but if it would seem to you that parents would most likely be hideously biased when assessing their children’s intellectual abilities even though, according to a giftedness researcher, this is probably not the case, then should you probably also consider that your concern that most LessWrong members are likely to subconsciously falsify their own IQ scores by a whopping 30 points (if that is your perception) may be far less likely to be a problem than you thought?
Scores on standardized tests like SAT and ACT can be improved via hard work and lots of practice—there are abundant practice books out there for such tests. It is entirely conceivable that those self-reported IQs were generated via comparing scores on these standardized tests against IQ-conversion charts. I.e., with very hard work, the apparent IQs are in the 130+ range according to these standardised tests; but when it comes to tests that measure your native intelligence (e.g., iqtest.dk), the scores are significantly lower. In future years, it would be advisable for the questionnaire to ask participants how much time they spent in total to prepare for tests such as SAT and ACT—and even then you might not get honest answers. That brings me to the point of lying...
Not necessarily true. If the survey results show that LWers generally have IQs in the gifted range, then it allows LWers to signal their intelligence to others just by identifying themselves as LWers. People would assume that you probably have an IQ in the gifted range if you tell them that you read LW. In this case, everyone has an incentive to fudge the numbers.
erratio has also pointed out that participants might have answered those personality tests untruthfully in order to signal intelligence, so I shan’t belabour the point here.
Ok, now here is a motive! I still find it difficult to believe that:
Most of 1000 people care so much about status that they’re willing to prioritize it over truth, especially since this is LessWrong where we gather around the theme of rationality. If there’s anyplace you’d think it would be unlikely to find a lot of people lying about things on a survey, it’s here.
The people who take the survey know that their IQ contribution is going to be watered down by the 1000 other people taking the survey. Unless they have collaborated by PM and have made a pact to fudge their IQ test figures, these frequently math oriented people must know that fudging their IQ figure is going to have very, very little impact on the average that Yvain calculates. I do not know why they’d see the extra work as worthwhile considering the expected amount of impact. Thinking that fudging only one of the IQs is going to be worthwhile is essentially falling for a Pascal’s mugging.
Registration at LessWrong is free and it’s not exclusive. At all. How likely is it, do you think, that this group of rationality-loving people has reasoned that claiming to have joined a group that anybody can join is a good way to brag about their awesomeness?
I suppose you can argue that people who have karma on their accounts can point to that and say “I got karma in a gifted group” but lurkers don’t have that incentive. All lurkers can say is “I read LessWrong.” but that is harder to prove and even less meaningful than “I joined LessWrong”.
Putting the numbers where our mouths are:
If the average IQ for lurkers / people with low karma on LessWrong is pretty close to the average IQ for posters and/or people with karma on LessWrong, would you say that the likelihood of post-making/karma-bearing LessWrongers lying on the survey in order to increase other’s status perceptions of them is pretty low?
Do you want to get these numbers? I’ll probably get them later if you don’t, but I have a pile of LW messages and a bunch of projects going on right now so there will be a delay and a chance that I completely forget.
From the public dataset:
165 out of 549 responses without reported positive karma (30%) self-reported an IQ score; the average response was 138.44.
181 out of 518 responses with reported positive karma (34%) self-reported an IQ score; the average response was 138.25.
One of the curious features of the self-reports is how many of the IQs are divisible by 5. Among lurkers, we had 2 151s, 1 149, and 10 150s.
I think the average self-response is basically worthless, since it’s only a third of responders and they’re likely to be wildly optimistic.
So, what about the Raven’s test? In total, 188 responders with positive karma (36%), and 164 responders without positive karma (30%) took the Raven’s test, with averages of 126.9 and 124.4. Noteworthy is the new max and min- the highest scorer on the Raven’s test claimed to get 150, and the three sub-100 scores were 3, 18, and 66 (of which I suspect only the last isn’t a typo or error of some sort).
Only 121 users both self-reported IQ and took the Raven’s test. The correlation between their mean-adjusted self-reported IQ and mean-adjusted Raven’s test was an abysmal .2. Among posters with positive karma, the correlation was .45; among posters without positive karma, the correlation was -.11.
Thank you for these numbers, Vaniver! I should have thanked you sooner. I had become quite busy (partly with preparing my new endless September post) so I did not show up to thank you promptly. Sorry about that.
You’re welcome!
I have thought of that. But a person who wants to lie about his IQ would think this way: If I lie and other LWers do not, it is true that my impact on the average calculated IQ will be negligible, but at least it will not be negative; but if I lie and most other LWers also lie, then the collective upward bias will lead to a very positive result which would portray me in a good light when I associate myself with other LWers. So there is really no incentive to not lie.
(I’m not saying that they definitely lied; I’m merely pointing out that this is something to think about.)
Fair point; but very often the kind of clubs you join does indicate something about your personality and interests, regardless of whether you are actually an active/contributing member or not. Saying “I read LessWrong” or “I joined LessWrong” certainly signals to me that you are more intelligent than someone who joined, say, Justin Bieber’s fan club, or the Twilight fan-fiction club. And if there are numbers showing that LW readers tend to have IQs in the gifted range, naturally I would think that X is probably quite intelligent just by virtue of the fact that X reads LW.
One last point is that LWers might not be deliberately lying: Perhaps they were merely victim to the Dunning-Kruger effect when self-reporting IQs. I am not sure if there are any studies showing that intelligent people are generally less likely to fall prey to the Dunning-Kruger effect.
Last but not least, I would again like to suggest that future surveys include questions asking people how much time they spent on average preparing for exams such as the SAT and the ACT—as I pointed out previously, scores on such exams can be very significantly improved just by studying hard, whereas tests like iqtest.dk actually measure your native intelligence.
Not true. It would probably take at least 20 minutes to fudge all the stuff that has to be fudged. When you’re already fatigued from filling out survey questions, that’s even less desirable at that time. At best, this would be falling for a Pascal’s mugging. True that some people may. But would the majority of survey participants… at a site about rationality?
They were not asked to assess their own IQ they were asked to report the results of a real assessment. To report something other than the results of a real assessment is a type of lie in this case.
That’s a suggestion for Yvain. I don’t assist with the surveys.
Make a copy and post it. Most browsers have the ability to print/save pages as PDFs or various forms of HTML.
Ok I managed to dig it up!
From the December 1993 Mensa Bulletin.
* The LessWrongers were added by me, using the same calculation method as in the comment where I test my personality type predictions and are based on the 2012 survey results.
Thanks for the analysis. I agree with your conclusion.
On a less relevant note, it does feel good to see more evidence that the community we hang out with is smart and awesome.
This also explains a lot of things. People regard IQ as if it is meaningless, just a number, and they often get defensive when intellectual differences are acknowledged. I spent a lot of time doing research on adult giftedness (though I’m most interested in highly gifted+ adults) and, assuming the studies were done in a way that is useful (I’ve heard there are problems with this), and my personal experiences talking to gifted adults are halfway decent as representations of the gifted adult population, there are a plethora of differences that gifted adults have. For instance, in “You’re Calling Who A Cult Leader?” Eliezer is annoyed with the fact that people assume that high praise is automatic evidence that a person has joined a cult. What he doesn’t touch on is that there are very significant neurological differences between people in just about every way you could think of, including emotional excitability. People assume that others are like themselves, and this causes all manner of confusion. Eliezer is clearly gifted and intense and he probably experiences admiration with a higher level of emotional intensity than most. If the readers of LessWrong and Hacker News are gifted, same goes for many of them. To those who feel so strongly, excited praise may seem fairly normal. To all those who do not, it probably looks crazy. I explained more about excitability in the comments.
I also want to say (without getting into the insane amount of detail it would take to justify this to the LW crowd—maybe I will do that later, but one bit at a time) that in my opinion, as a person who has done lots of reading about giftedness and has a lot of experience interacting with gifted people and detecting giftedness, the idea that most survey respondents are giving real answers on the IQ portion of the survey seems very likely to me. I feel 99% sure that LessWrong’s average IQ really is in the gifted range, and I’d even say I’m 90%+ sure that the ballpark hit on by the surveys is right. (In other words, they don’t seem like a group of predominantly exceptionally or profoundly gifted Einsteins or Stephen Hawkings, or just talented people at the upper ends of the normal range with IQs near 115, but that an average IQ in the 130′s / 140′s range does seem appropriate.)
This says nothing about the future though… The average IQ has been decreasing on each survey for an average of about two points per year. If the trend continues, then in as many years as LessWrong has been around, LessWrong may trend so far toward the mean that LessWrong will not be gifted anymore (by all IQ standards that is, it would still be gifted by some definitions and IQ standards but not others). I will be writing a post about the future of LessWrong very soon.
Would you predict then that people who’re not gifted are in general markedly less inclined to praise things with a high level of intensity?
This seems to me to be falsified by everyday experience. See fan reactions to Twilight, for a ready-to-hand example.
My hypothesis would simply be that different people experience emotional intensity as a reaction to different things. Thus, some think we are crazy and cultish, while also totally weird for getting excited about boring and dry things like math and rationality… while some of us think that certain people who are really interested in the lives of celebrities are crazy and shallow, while also totally weird for getting excited about boring and bad things like Twilight.
This also leads each group to think that the other doesn’t get similar levels of emotional intensity, because only the group’s own type of “emotional intensity” is classified as valid intensity and the other group’s intensity is classified as madness, if it’s recognized at all. I’ve certainly made the mistake of assuming that other people must live boring and uninteresting lives, simply because I didn’t realize that they genuinely felt very strongly about the things that I considered boring. (Obligatory link.)
(Of course, I’m not denying there being variation in the “emotional intensity” trait in general, but I haven’t seen anything to suggest that the median of this trait would be considerably different in gifted and non-gifted populations.)
Ok, where do I find them?
If you have to go looking, you’re lucky.
If you want to find them in person, the latest Twilight movie is still in theaters, although you’ve missed the people who made a point of seeing it on the day of the premier.
Haha, I guess so. I am very, very nerdy. I had fun getting worldly in my teens and early 20′s, but I’ve learned that most people alienate me, so I’ve isolated myself into as much of an “ivory tower” as possible. (Which consists of me doing things like getting on my computer Saturday evenings and nerding so hard that I forget to eat.)
Not really.
What did they do when you saw them?
How do we distinguish the difference between the kind of fanaticism that mentally unbalanced people display for, say, a show that is considered by many to have unhealthy themes and the kind of excitement that normal people display for the things they love? Maybe Twilight isn’t the best example here.
I didn’t. I don’t particularly have to go out of my way to find Twilight fans, but if I did, I wouldn’t.
I think you’re dramatically overestimating the degree to which fans of Twilight are psychologically abnormal. Harlequin romance was already an incredibly popular genre known for having unhealthy themes. Twilight, like Eragon, is a mostly typical work of its genre with a few distinguishing factors which sufficed to garner it extra attention, which expanded to the point of explosive popularity as it started drawing in people who weren’t already regular consumers of the genre.
I wouldn’t be surprised if this is true.
This still does not answer the question “What sample can we use that filters out fanaticism from mentally unbalanced people to compare the type of excitement that gifted people feel to the type of excitement that everyone else feels?” Not to assume that no gifted people are mentally unbalanced… I suppose we’d really have to filter those out of both groups.
Taboo “mentally unbalanced”.
What distinction are you trying to make here?
we will all be brain-dead in 70 years.
It’s true that the downward trend can’t go on forever, and to say that it’s definitely going to continue would be (all by itself, without some other arguments) an appeal to history or slippery slope fallacy. However, when we see a trend as consistent and as potentially meaningful as the one below, it makes sense to start wondering why it is happening:
IQ Trend Analysis
I was mostly just trying to point out that you are extrapolating from a sample size of three points. Three points which have a tremendous amount of common causes that could explain the variation. Furthermore you aren’t extrapolating 10% further from the span of your data, which might be ok, but actually 100% further. You’re extrapolating for as long as we have data, which is… absurd.
One, I am used to seeing the term “sample size” applied to things like the people being studied, not a number of points done in a calculation. If there is some valid use of the term “sample size” that I am not aware of would you mind pointing me in the correct direction?
Two, I am not sure where you’re getting “three points” from. If you mean the amount of IQ points that LessWrong has lost on the studies, then it was 7.18 points, not three.
Two points per year, which could be explained in other ways, sure. No matter what the trend, it could be explained in other ways. Even if it was ten points per year we could still say something like “The smartest people got bored taking the same survey over and over and stopped.” There are always multiple ways to explain data. That possibility of other explanations does not rule out the potential that LessWrong is losing intelligent people.
Not sure what these 10% and 100% figures correspond to. If I am to understand why you said that, you will have to be specific about what you mean.
Including all of the data rather than just a piece of the data is bad why?
Three points referred to the number of surveys taken, which I didn’t bother to look up, but I believe is three.
10% and 100% referred to the time span over which these data points referred to, ie. three years. Basically, I might be OK with you making a prediction for the next three months (still probably not) but extrapolating for three years based on three years of data seems a bit much to me.
Oh I see. The problem here is that “if the trend continues” is not a prediction. “I predict the trend will continue” would be a prediction. Please read more carefully the next time. You confused me quite a bit.
If you’re not making a prediction, then it’s about as helpful as saying “If the moon crashes into North America next year, LW communities will largely cease to exist.”
Looks like Aumann at work. My own readings, though more specifically on teenage giftedness in the 145+ range, along with stuff on ASD and asperger, heavily corroborate with this.
When I was 17, my (direct) family and I had strong suspicions that I was in this range of giftedness—suspicions which were never reliably tested, and thus neither confirmed nor infirmed. It’s still up in the air and I still don’t know whether I fit into some category of gifted or special individuals, but at some point I realized that it wasn’t all that important and that I just didn’t care.
I might have to explore the question a bit more in depth if I decide to return into the official educational system at some point (I mean, having a paper certifying that you’re a genius would presumably kind of help when making a pitch at university to let you in without the prerequisite college credit because you already know the material). Just mentioning all of the above to explain a bit where my data comes from. Both of my parents and myself were all reading tons of books, references, papers and other information along with several interviews with various psychology professionals for around three months.
Also, and this may be another relevant point, the only recognized, official IQ test I ever took was during that time, and I had a score of “above 130”² (verbal statement) and reportedly placed in the 98th and 99th percentiles on the two sections of a modified WAIS test. The actual normalized score was not included in the report (that psychologist(?¹) sucked, and also probably couldn’t do the statistics involved correctly in the first place).
However, I was warned that the test lost statistical significance / representativeness / whatever above 125, and so that even if I had an IQ of 170+ that test wouldn’t have been able to tell—it had been calibrated for mentally deficient teenagers and very low IQ scores (and was only a one-hour test, and only ten of the questions were written, the rest dynamic or verbal with the psychologist). Later looking-up-stats-online also revealed that the test result distributions were slightly skewed, and that a resulting converted “IQ” of “130″ on this particular test was probably more rare in the general population than an IQ of 130 normally represents, because of some statistical effects I didn’t understand at the time and thus don’t remember at all.
Where I’m going with this is that this doesn’t seem like an isolated effect at all. In fact, it seems like most of North America in general pays way more attention to mentally deficient people and low IQs than to high-IQs and gifted individuals. Based on this, I have a pretty high current prior that many on LW will have received scores suffering from similar effects if they didn’t specifically seek the sorts of tests recommended by Mensa or the likes, and perhaps even then.
Based on this, I would expect such effects to compensate or even overcompensate for any upward nudging in the self-reporting.
=====
I don’t know if it was actually a consulting psychologist. I don’t remember the title she had (and it was all done in French). She was “officially” recognized to be in legal capacity to administrate IQ tests in Canada, though, so whatever title is normally in charge of that is probably the right one.
Based on this, the other hints I mention in the text, and internet-based IQ tests consistently giving me 150-ish numbers when at peak performance and 135-ish when tired (I took those a bit later on, perhaps six months after the official one), 135 is the IQ I generally report (including in the LW survey) when answering forms that ask for it and seems like a fairly accurate guess in terms of how I usually interact with people of various IQ levels.
Was Mensa’s test conducted on the internet? The internet has a systematic bias in personalities. For example, reddit subscriptions to each personality type reddit favor Introversion and Intuition 4,828 INTJ 4,457 INTP 1,817 INFP 1,531 INFJ
IAWYC, but “the internet” is way too broad for what you actually mean—ISTM that a supermajority of teenagers and young adults in developed countries uses it daily, though plenty of them mostly use it for Facebook, YouTube and similar and probably have never heard of Reddit. (Even I never use Reddit unless I’m following a link to a particular thread from somewhere else—but the first letter of my MBTI is E so this kind of confirms your point.)
Yeah...by “internet” what I meant was sites that most people do not know about—sites that you would only stumble upon in the course of extensive net usage. I once described it to a friend as “deep” vs “shallow” internet, with depth corresponding to the extent to which a typical visitor to the website uses the internet. Even within a website (say reddit) a smaller sub-reddit would be “deeper” than a main one.
I’m myself am actually a counterexample to my own “extroverts don’t use the internet as much” notion...but I’m only a moderate extrovert. (ENTP or ENFP depending on the test...ENTP description fits better. I listed ENTP in the survey.)
By that definition, there are many nearly disconnected “deep internets”.
Yes...i’m confused. Is this supposed to be a flaw in the definition? The idea here is to use relative obscurity to describe the degree to which a site is visited only by Internet users who do heavy exploring. There are only a few “shallow” regions… Facebook, Wikipedia, twitter...the shallowest being google. These are all high traffic and even people who never use computers have heard some of these words. There are many deep regions, on the other hand, and most are disconnected.
It is if you then proceed to claim to have statistics over users of the “deep internet”.
Yeah, different websites have different personality skews, which complicates things. Fortunately there’s evidence against Mensa having used an online sample: Epiphany said the results were published in December 1993. It’s fairly easy to give a survey to an Internet forum nowadays, but where would Mensa have found an online sample back in ’93? IRC? Usenet? (There is a rec.org.mensa where people posted about personality and the Myers-Briggs back in 1993, but the only relevant post that year was someone asking about Mensans’ personalities to no avail.)
I don’t have any more data than that, sorry.
To suggest that people on the internet may have certain personality types is a good suggestion, but it raises two questions:
Might your example of Reddit be similar to LW because LW gets lots of users from Reddit? (Or put another way, if the average LessWronger is gifted, maybe “the apple doesn’t fall far from the tree” and Reddit has lots of gifted people, too.)
Might gifted people gather in large numbers on the internet because it’s easier to find people with similar interests? (Just because people on the internet tend to have those personality types, it doesn’t mean they’re not gifted.)
As for “the internet” having a systematic bias in personalities, I would like to see the evidence of this that’s not based on a biased sample. It’s likely that the places you go to find people like you will, well, have people like you, so even if you (or somebody else on one of those sites) observed a pattern in personality types across sites they hang out on, the sample is likely to be biased.
I’d say “LW has about as many gifted people as Reddit (proportionally)” should be a sort of null hypothesis: if this is true, then people on LessWrong are not actually surprisingly smart.
I wouldn’t say that’s a reasonable null. Reddit has like 8 million users; 2% of the 310m American population is just 6.2m, so it would be difficult for Reddit to be 100% gifted while LW could easily be. The size disparity is so large that such a null seems more than a little weird.
I don’t think I understand your objection. If LW were 100% gifted (while Reddit, presumably, is not?) wouldn’t that be evidence that there’s some sort of IQ selection at work? (or, conceivably, that just being on LW makes people smarter, although I think that’s not supposed to be a thing).
I’m saying that we could, just from knowing how big Reddit is, reject out of hand all sorts of proportions of gifted because it would be nigh impossible; a set of nulls (the proportions 0-100%), many of which (all >75%) we can reject before collecting any data is a pretty strange choice to make!
Well, really what I want to ask is: is LW any different, IQ-wise, from a random selection of Redditors of the same size? Possibly stating it in terms of a proportion of “gifted” people is misleading, but that’s not as interesting anyway.
I don’t see the difference. A random selection of Redditors is going to depend on what Reddit overall looks like...
Well, I don’t see the difference either, but I’m still not entirely sure what about this hypothesis seems unreasonable to you, so I was hoping this reformulation would help.
The reasoning behind it is as follows: I figure a generic discussion board on the Internet has roughly the same IQ distribution as Reddit. If LW has a high average IQ, but so does Reddit, then presumably these are both due to the selection effect of “someone who posts on an online discussion board”. So to see if LW is genuinely smarter, we should be comparing it to Reddit, not to the Normal(100,15) distribution.
I would be shocked if that were true. Even after having grown stupendously, Reddit is still better than most discussion boards I happen to read.
Okay, fair enough. I don’t actually have much experience with Reddit.
I still think it’s a reasonable reference class. For one thing, LW runs on Reddit-based code. In particular, I would say that being significantly smarter than Reddit is a good cutoff for the feeling of smugness to start kicking in.
Maybe it just means Reddit-folk are surprisingly smart? I mean, IQ 130 corresponds to 98th percentile. The usual standard for surprise is 95th percentile.
That’s a good point—I hadn’t considered sample bias. Extending that point, though, Lesswrong and Mensa are a biased sample in more than the simple fact that the people are gifted. It is only a subset of gifted people that choose to participate in Mensa It should be mentioned, I’m using “internet” as shorthand for the “deep” internet … not facebook. I’m talking websites that most people do not use, that you’d have to spend a lot of time on the internet to find. As such, the “internet” hypothesis would predict a greater bias towards smaller sub-reddits.
Anyway, I was mostly posing an alternate hypothesis. When I first noticed the trend on the personality forums, this is what I thought was happening -
Slacking off / internet addiction selects for Perceiving and low Conscientiousness.
Non-social-networking internet use selects for Introversion.
Any forum discussing an idea without immediate practical benefits selects for iNtuition.
And then, factor in lesswrong/giftedness...
If it’s a math/science/logic topic, it selects for Thinking and iNtuition.
High scores on Raven’s matrices select for Thinking, iNtuition. High scores on Working memory components select for Judging. The ACT/SAT additionally select for Conscientiousness
Strong mathematical affinity shifts those on the border of NTP and NTJ into *NTJ (people prefer dealing with intellectually ordered systems, even if they have messy rooms and chaotic lifestyles)
A scientific/engineering ideology creates a shift towards the concrete (empirical evidence, practical gains in technology, etc) shifts those on the border of NTJ and STJ into ISTJ.
In summary, I think LW and Mensa surveys are attracting a special subset of idea driven and logical people (iNtuitives and Thinkers) and likely to use the internet often/spot the survey. (Introverts)
That’s much nicer and much more detailed. Questions this raises:
Might the “deep” internet you refer to be selecting for gifted people? (I think this is likely!)
Do we have figures on personality types and IQs for internet forums in general, not from a biased sample set? These figures would test your theory.
I agree with (1), but would claim that it also selectively attracts introverts (and I’m unsure whether or not it will bias J-P to the P side)
(2) For each of these, I tried not to look at the data after finding the poll. I made predictions first. Just for fun / to correct for hindsight bias, anyone reading might want to do the same. To play, don’t click on the link or read my prediction until you make yours. Also, here is some data which claims to represent the general population—http://mbtitruths.blogspot.com/2011/02/real-statistics.html for comparison. I’ve already seen similar data on another site, so I won’t state my predictions on this one.
A website posts stats for people who have taken the test. Unlike the above simple random sample, this selects for internet users.
http://www.personalitypage.com/html/demographics.html
Prediction: I’d consider this “shallow internet”, so very weak biases to (I). The general population is (S), I’d expect a weak bias to (N) but not enough to overcome the general population’s S centering.
Result: apparently I suck at predictions. In hindsight all the top three would be predicted score high “Fi” on a Jungian cognitive function test, and Fi in theory would be more interested in taking personality tests. But that’s hindsight, and I’m not sure if connection between MBTI and Jung hasn’t been verified empirically.
Here is a “deep internet” forum that I wouldn’t ever visit… Christian singles chat forum! This should not suffer from the sample bias you mentioned earlier (He stated that websites I visit are likely to have users with similar personalities to me [ENTP])
http://christianchat.com/christian-singles-forum/34516-meyers-briggs-type-indicator-mbti-poll.html
Prediction: I tried my best not to look at the data despite the high visual salience as soon as you open that link. Here is my prediction: I’d predict strong biases towards Introversion (because internet), slight biases towards iNtuition (because religion is idea-based), moderate bias to Feeling (I think religious people are illogical) and … let’s say a slight tilt towards Judging. Call it a hunch, life experience says that Si (judging + sensing) is particularly predisposed to religion.
Result: OK, looks like my trends were right but my magnitude was way off. My “hunch” was correct but I didn’t listen to it closely enough and vastly underestimated the Judging bias, while my personal prejudice overestimated the Feeling bias. My predictions about intution and introversion were essentially correct though.
http://personalitycafe.com/myers-briggs-forum/28171-mbti-demographics.html
Click the ppt, it has data by education.
Prediction: NT’s pursue higher education, SF’s do not. Other two dichotomies don’t matter as much, but J helps slightly.
Result: seems about right. Eyeballing, J seems not to matter much until college, at which point it prevents dropping out.
For IQ—http://asm.sagepub.com/content/3/3/225.short
Prediction- Strong N, slight T bias. I don’t think T actually means “intelligent” as I define it, but I do think it would help on some portions of the IQ test.
Result: N bias only. interesting.
Finally, Scientific aptitude: http://www.amsciepub.com/doi/pdf/10.2466/pr0.1970.26.3.711
Prediction—strong N, moderate T. I’m not sure about J-P. I think people who choose science tracks and go into academia will be P (creative types), whereas kids who get good grades but ultimately do not choose science will be J. I’m not sure which group they are looking at (I didn’t permit myself to read it yet, so I’m a bit vague on what exactly they did). I don’t think E—I will matter at all.
Result -
NT take high level science a lot more, Introverts take them slightly more. J-P is irrelevant. Intuition really helps in school at all levels. Feeling relates to high GPA in the easy courses but not the hard course (that’s pretty unexpected) Introversion relates to high GPA in hard course but not in easy courses. Percievers start out with a pretty big edge both in IQ and GPA in the lower level courses, Judging takes a slight lead in both those metrics in the advanced course. Not sure if this is noise.
Side finding—they also did IQ measurements. Again, only N related to IQ (in fact, F won out over T)...but it did not relate as much in the advanced courses. I think the advanced course chopped off the lower end of the IQ bell curve, leaving only smart Sensors. By the way, Extroverts have an IQ edge, despite getting lower grades and not taking advanced courses as often.
Thoughts? I think in general my ideas about introversion not mattering for intelligence, but mattering a lot for internet use, bear out. Apparently Thinking doesn’t really matter either...which I sort of felt was true, but I didn’t actually expect the IQ test scores to agree with me on that. It might have to do with self reported vs actual use of logic.
Of course, we are looking at the center of the bell curve, whereas on LW we are (presumably) looking at the far right edge.
EDIT: here is another IQ one with bigger sample size. http://www.psytech.com/Research/Intelligence-2009-08-11.pdf
They say that they found IQ correlates with I, N, T, and P. However, they claim that were surprised about the “I” correlation, because a large number of other studies have found that E is positively correlated. They go on to talk about how different testing conditions might favor E vs I. Some interesting further reading in there...it seems that N only correlates on the verbal reasoning section,
I’m inclined to believe the survey results myself, but there is a third possibility. If a certain personality type (or distribution of types) reflects a desire to associate with gifted people, or to be seen as gifted, we’d likely expect that to be heavily overrepresented in MENSA; that’s pretty much the reason the club exists, after all. We might also expect people with those desires to be less inclined to share average or poor IQ results, or even to falsify results.
If the same personality type is overrepresented here, then we have a plausible cause for similar personality test results and for exaggerated IQ reporting, without necessarily implying that the actual IQ distributions are similar.
Looking at Groups of IQs:
I acknowledge that the sample set for the highest IQ groups are, of course, rather small, but that’s all we’ve got. What’s been happening with the numbers for the highest IQ groups, if indicative of what’s really happening, is not encouraging. The highest two groups have decreased in numbers while the lowest two have increased. Also, it looks like the prominence of each group has shifted over time such that the highest group went from being 1⁄5 to 1⁄20 and the moderately gifted and normal groups have grown substantially.
Exceptionally Gifted Respondents (Self-Reported IQ)
(Defined as having an IQ of 160 or more)
2009: 11 (7%)
2011: 27 (3%)
2012: 22 (2%) (Decreased)
Highly Gifted Respondents (Self-Reported IQ)
(Defined as having an IQ between 145-159)
2009: 17 (11%)
2011: 88 (9%)
2012: 81 (7%) (Decreased)
Moderately Gifted Respondents (Self-Reported IQ)
(Defined as having an IQ between 132-144)
2009: 22 (14%)
2011: 125 (13%)
2012: 149 (11%) (Increased)
Normal Respondents (Self-Reported IQ)
(Defined as having an IQ between 100-131)
2009: 11 (7%)
2011: 91 (10%)
2012: 94 (9%) (Increased)
Each Group as a Percentage of Total IQ Respondents, by Year:
2009 Group IQ Distribution (As a percentage of 61 total IQ respondents)
18% Exceptionally Gifted
28% Highly Gifted
36% Moderately Gifted
18% Normal IQ
2011 Group IQ Distribution (As a percentage of 331 total IQ respondents)
8% Exceptionally Gifted
27% Highly Gifted
38% Moderately Gifted
28% Normal IQ
2012 Group IQ Distribution (As a percentage of 346 total IQ respondents)
6% Exceptionally Gifted
23% Highly Gifted
43% Moderately Gifted
27% Normal IQ
I don’t find it that hard to see why Lesswrong and Mensa would both select for introverted personalities. Do you?
I think most sensible people can deduce that IQ is positively correlated with SAT and ACT and all of them are positively correlated with “status”. I agree that SAT and ACT are more difficult to fudge though. I haven’t ever done either of them. Can they be easily redone several times? Do (smart) people liberally talk about their scores in the US?
Many people do IQ tests of different calibers several times and could just remember or report the best result they’ve gotten. There are different levels of dishonesty. “Lying” is a bit crude.
I don’t think anyone on Less Wrong has lied about their IQ. (addendum: not enough to seriously alter the results, anyway.) If you come up with a “valuing the truth” measure, LessWrong would score pretty highly on that considering the elaborate ways people who post here go about finding true statements in the first place. To lie about your IQ would mean you’d have to know to some degree what your real IQ is, and then exaggerate from there.
However, I do think it’s more likely than you mention that most people on LessWrong self-reporting IQ simply don’t know what their IQ is in absolutely certain terms, since to know your adult IQ you’d have to see a psychometricist and receive an administered IQ test. iqtesk.dk is normed by Mensa Denmark, so it’s far more reliable than self-reports. You don’t know where the self-reported IQ figures are coming from—they could be from a psychometricist measuring adult IQ, or they could be from somewhere far less reliable. It could be that they know their childhood IQ was measured at somewhere around 135 for example, and are going by memory. Or they could know by memory that their SAT is 99th percentile and spent a minute to look up what 99th percentile is for IQ, not knowing it’s not a reliable proxy. Or they might have taken an online test somewhere that gave ~140 and are recalling that number. Who knows? Either way, I consider “don’t attribute to malice what you can attribute to cognitive imperfection” a good mantra here.
126 is actually higher than a lot of people think. As an average for a community, that’s really high—probably higher than all groups I can think of except math professors, physics professors and psychometricists themselves. It’s certainly higher than the averages for MIT and Harvard, anyway.
About the similarity between self-reported IQ and SAT scores: SAT scores pre-1994 (which many of the scores on here are not likely to fall into) are not reliable as IQ test proxies; Mensa no longer accepts them. This is because it is much easier to game. I tutor the SAT, and when I took the SAT prior to applying at a tutor company my reading score was 800, but in high school pre-college it was only in the mid-600s. SAT scores in reading are heavily influenced by (1) your implicit understanding of informal logic, and (2) your familiarity with English composition and how arguments/passages may be structured. Considering the SAT has contained these kinds of questions since the mid-90s, I am inclined to throw its value as a proxy IQ test out the window and don’t think you can draw conclusions about LessWrong’s real collective IQ from the reported SAT scores.
The IQTest.dk result may have given the lowest measure, but I also think it’s the most accurate measure. It would not put LessWrong in the 130s, maybe, but it would mean that the community is on the same level of intellect as, say, surgeons and Harvard professors, which is pretty formidable for a community.
Over 1000 people took the test. Statistically speaking, it should have included about 50 sociopaths. Not all sociopaths would necessarily lie on that question but considering that you’re going to have to explain why you think that none of the sociopaths lied (or pathological liars or borderlines or other types that are likely to have been included in the test results) you have chosen a position, or at least wording, which is going to be darned near impossible for you to defend.
No because to say “I know my IQ” when one doesn’t is also a lie, and that’s what it would be saying if they put ANY IQ in the box without knowing it.
Mensa is a club not a professional IQ testing center. They’re not even legally allowed to give out scores anymore. Their test scores are not considered to be accurate. For one thing, they (and iqtest.dk) do not evaluate for learning disorders. One in six gifted people has a learning disorder. Learning disorders lower one’s score and so the test should be adjusted to reflect this.
The iqtest.dk scores ARE self-reported. That is to say, the user types the IQ score into the survey box themselves. In that way, they’re equally flawed to the other intelligence questions, not “more reliable than self-reports”.
I stopped here because the rest of the comment follows the same pattern. About every other sentence in your comment is irrational. LessWrong is going to eat you alive, honey. Get out while you’re ahead.
Not if LessWrong values truthseeking activities more than the general population, or considers lying/truth-fabrication a greater sin than the general population does, or if LessWrong just generally attracts less sociopaths than the general population. If over 1000 fitness enthusiasts take a test about weight, the statistics re: obesity are not going to reflect the general population’s. Considering the CRT scores of LessWrong and the nature of this website to admire introspection and truthseeking activities, I doubt that LW would be reflective of the general population in this way.
Lies are more than untrue statements; at least, in the context of self-reports, they are conscious manipulations of what one knows to be true. Someone might think they know their IQ because they’ve taken less reliable IQ tests, or because they had a high childhood IQ, or because they extrapolated their IQ from SAT scores, or for a host of other reasons. In this case they haven’t actually lied, they’ve just stated something inaccurate.
Someone could put an IQ when they have no idea what their IQ is, yes, in the sense that they have never taken a test of any sort and have no idea what their IQ would be if they took one, even an inaccurate one. I don’t think many people here would do that, though, because of the truthseeking reasons mentioned earlier.
Mensa doesn’t need to be a professional IQ testing center for their normings to be accurate, however. I am also not sure how not accounting for learning disorders would seriously alter IQTest.dk’s validity over self-reports.
However, it’s inaccurate to say that because someone puts their number in the box from IQTest.dk that they’re “equally flawed” to the other intelligence questions. Someone who self-reports an IQ number, any number, may not know if that number was obtained using accurate methodology. It may be an old score from childhood, and childhood IQ scores vary wildly compared to adult IQ scores. It may be an extrapolation from SAT scores, as I mentioned above. There are a number of ways in which self-reported IQ differs from reported IQtest.dk IQ.
This reads as unnecessarily tribalistic to me. I take it you think I am an undiscriminating skeptic? In any case, cool it.
I’d expect Less Wrongers to be more likely to be sociopaths than average. We’re generally mentally unusual.
Yeah, I am perfectly aware that the IQ score I got when I was three wasn’t valid then and certainly isn’t now. The survey didn’t ask “What’s a reasonable estimate of your IQ?”.
That is why I used the wording “statistically speaking”—it is understood to mean that I am working from statistics that were generated on the overall population as opposed to the specific population in question. You are completely ignoring my point which is that you have chosen a position which is going to be more or less impossible to defend. That position was:
It’s considered very rude to completely ignore someone’s argument and nit pick at their wording. That is what you just did.
Now it’s like you’re trying to make up a new definition of the word lying so you can continue to think your ridiculous assessment that:
By the common definition of the word “lie” producing a number when you do not know the number definitely does qualify as a lie. You’re not fooling me by trying to make a new definition of the word “lie” in this context. This behavior just looks ridiculous to me.
But they do need to provide a professional IQ testing service if they want their norms to mean something. The iqtest.dk might turn out to be a better indicator of visual-spatial ability than IQ, or it might discriminate against autistics, which LW might have an unusually large number of (seeing as how there are a lot of CS people here).
Here you go twisting my wording. I specifically said:
The only reason I’m responding to you is because I am hoping you will see that you need to do more work on your rationality. Please consider getting some rationality training or something.
The general population would contain 50 sociopaths to 1000; I don’t think LessWrong contains 50 sociopaths to 1000. Rationality is a truth-seeking activity at its core, and I suspect a community of rationalists would do their best to avoid lying consciously.
I am not sure what “the common definition of the word ‘lie’” is, especially since there are a lot of differing interpretations of what it means to lie. I know that wrong answers are distinct from lies, however. I think that a lot of LessWrong people might have put an IQ that does not reflect an accurate result. But I doubt that many LessWrong people have put a deliberately inaccurate result for IQ. Barring “the common definition” (I don’t know what that is), I’m defining “stating something when you know what you are stating is false” as a lie, since someone can put a number when they don’t know for sure what the true number is but don’t know that the number they are stating is false either.
I don’t know what you mean by “mean something” with respect to Mensa Denmark’s normings. They will probably be less accurate than a professional IQ testing service, but I don’t know why they would be inaccurate or “meaningless” by virtue of their organization not being a professional IQ testing service.
The only way I can think of in which the self-reported numbers would be more accurate than the IQTest.dk numbers is if the LW respondents knew that their IQ numbers were from a professional testing service and they had gone to this service recently. But since the test didn’t specify how they obtained this self-report, I can’t say, nor do I think it’s likely.
IQTest.dk uses Raven’s Progressive Matrices which is a standard way to measure IQ across cultures. This is because IQ splits between verbal/spatial are not as common. It wouldn’t discriminate against autistics, because it actually discriminates in favor of autistics; people with disorders on the autism spectrum are likely to score higher, not lower.
I’m not sure how the bolding of “in that way” bolsters your argument. Paraphrased, it would be “in the way that the user types the IQ score into the survey box themselves, the IQTest.dk questions are equally flawed to the other intelligence questions.” But this neglects to consider that the source of the number is different; they are self-reports in the sense that the number is up to someone to recall, but if someone types in their IQTest.dk number you know it came from IQTest.dk. If someone types in their IQ without specifying the source, you have no idea where they got that number from—they could be estimating, it could be a childhood test score, and so on.
Remarks like these are unnecessary, especially since I’ve just joined the site.
In principle, one could make up a number or insert a number other than what they got. But I don’t think a nontrivial fraction of respondents did that.
Do you have statistics about how many sociopaths take extra-long online tests or how many sociopaths frequent rationalist forums? Or are you just talking about the percentage of sociopaths in the general population?
As a sidenote one would think that people willing to lie about their IQ would be positively correlated with people that look up Bayes’ birthdate before filling in their ‘estimation’. Anyone making a statistical analysis regarding this?
Downvoted for this statement and overall unnecessary rude tone.
Perhaps you became busy or something and did not have a chance to respond to my comment, but I am still curious about this:
I agree with that article and that’s exactly why I downvoted you. You were contemptuously calling someone ‘honey’ and behaving like an all-around dick that smirks at newcomers and warns them that the rest of us will chew them up. That’s not what I want to see belong here and I won’t be a pacifist about it when you’re behaving like a weed.
I read the article again very carefully, trying to figure out whether Eliezer was advocating weeding people for behaving the way that I did. The article is about keeping “fools” out of the “garden of intelligent discussion”. It says nothing about the tone of posts and what tones should be weeded.
Actually, that was intended to make the tone friendlier. I acknowledge that this is not the way that you perceived it. My feeling is not contempt. I just don’t think he is likely to contribute constructively.
I like newcomers. The sole problem here was that about half of what this specific newcomer said was irrational.
That is exactly how I felt when I saw alfredmacdonald posting a bunch of irrational thoughts in this place for rationality. Are we both doing something wrong, then? The tone of your last comment doesn’t look any different to me than the tone of my comments. Is it that you feel that using a tone like this is never justified and we’ve both made a mistake or is it that you believe it’s okay to speak like this to people you feel are rude, but not to people you think are being irrational?
I speak to you like this because the simple explanation of “I downvoted you for excessive rudeness” doesn’t seem to satisfy you, to the point that you keep asking me for further clarification (and re-asked me when I ignored your first question). So I have to change my tone, because though the repetition of the same clarification “I downvoted you for excessive rudeness” should be adequate, you don’t get it.
Let me mention that I won’t continue discussing this, and if you continue pestering me you’ll be incentivize me to not offer any clarification at all for future downvotes towards your person, to just downvote you without explanation.
I see that you’re not interested in discussing the original issue that started this. I know that everyone has limited energy, so I accept this. It feels important to mention that none of my comments were written with an intent to pester you. I am not disagreeing with you about how you experienced them—different people experience things differently. I only mean to tell you that I did not intend to cause you this experience.
My intent was to understand your point of view better to see if our disagreement over whether a cold tone is justified for the purpose of garden-keeping would be resolved or if I would learn anything.
I hope you can see that despite our disagreement about how to protect the quality of the discussion area, we both care about whether the quality of the discussion area is good, and are willing to take action to protect it. I am not trying to troll; this wasn’t for “lulz”. I am doing it because I care. We have that one thing in common.
For this reason, I would prefer to use a friendly or neutral tone with you in the future. You may or may not be interested in putting this difference aside in order to have smoother interactions in the future, but I am willing to, so I invite you to do the same.
What do you say?
The verbiage “statistically speaking” was supposed to imply an acknowledgement that I know that the statistics were based on the overall population, not the specific context.
Ooh. This is a very, very good point. And if the survey participants really wanted to look gifted, they’d have probably decided that fudging the Bayes question was a necessity. I added your thought to my IQ accuracy comment. Upvote.
Thank you for explaining your downvote.
Is it that you disagree with Well-Kept Gardens Die By Pacifism or that you do this in a different way? If you have some different method, what is it?
Note: It’s probably inevitable that someone will ask me why I seem to agree with the spirit of this article, if I don’t believe in “elitism”. My answer is, succinctly, that humans seek like-minded people to hang out with, that this is part of fulfilling one’s social needs, and that it’s silly to allow our attempt to get basic needs met to be politicized and called “elitism” just because we gather around intellectualism when it is no different from the desire of a single mom to spend some time out with adults because children can’t have the same conversations or the desire of hunting-minded people to engage in activities without vegetarians harping on them (or vice versa).