Against Individual IQ Worries

Scott Alexander28 Sep 2017 17:12 UTC
69 points
8 comments8 min readLW link

[Related to: Attitude vs. Altitude]

I.

I write a lot about the importance of IQ research, and I try to debunk pseudoscientific claims that IQ “isn’t real” or “doesn’t matter” or “just shows how well you do on a test”. IQ is one of the best-studied ideas in psychology, one of our best predictors of job performance, future income, and various other forms of success, etc.

But every so often, I get comments/​emails saying something like “Help! I just took an IQ test and learned that my IQ is x! This is much lower than I thought, and so obviously I will be a failure in everything I do in life. Can you direct me to the best cliff to jump off of?”

So I want to clarify: IQ is very useful and powerful for research purposes. It’s not nearly as interesting for you personally.

How can this be?

Consider something like income inequality: kids from rich families are at an advantage in life; kids from poor families are at a disadvantage.

From a research point of view, it’s really important to understand this is true. A scientific establishment in denial that having wealthy parents gave you a leg up in life would be an intellectual disgrace. Knowing that wealth runs in families is vital for even a minimal understanding of society, and anybody forced to deny that for political reasons would end up so hopelessly confused that they might as well just give up on having a coherent world-view.

From an personal point of view, coming from a poor family probably isn’t great but shouldn’t be infinitely discouraging. It doesn’t suggest that some kid should think to herself “I come from a family that only makes $30,000 per year, guess that means I’m doomed to be a failure forever, might as well not even try”. A poor kid is certainly at a disadvantage relative to a rich kid, but probably she knew that already long before any scientist came around to tell her. If she took the scientific study of intergenerational income transmission as something more official and final than her general sense that life was hard – if she obsessively recorded every raise and bonus her parents got on the grounds that it determined her own hope for the future – she would be giving the science more weight than it deserves.

So to the people who write me heartfelt letters complaining about their low IQs, I want to make two important points. First, we’re not that good at measuring individual IQs. Second, individual IQs aren’t that good at predicting things.

II.

Start with the measurement problems. People who complain about low IQs (not to mention people who boast about high IQs) are often wildly off about the number.

According to the official studies, IQ tests are rarely wrong. The standard error of measurement is somewhere between 3-7 points (1, 2, 3). Call it 5, and that means your tested IQ will only be off by 5+ points 32% of the time. It’ll only be off by 10+ points 5% of the time, and really big errors should be near impossible.

In reality, I constantly hear about people getting IQ scores that don’t make any sense.

Here’s a pretty standard entry in the “help my IQ is so low” genre – Grappling With The Reality Of Having A Below Average IQ:

When I was 16, as a part of an educational assessment, I took both the WAIS-IV and Woodcock Johnson Cognitive Batteries. My mother was curious as to why I struggled in certain subjects throughout my educational career, particularly in mathematical areas like geometry.
I never got a chance to have a discussion with the psychologist about the results, so I was left to interpret them with me, myself, and the big I known as the Internet – a dangerous activity, I know. This meant two years to date of armchair research, and subsequently, an incessant fear of the implications of my below-average IQ, which stands at a pitiful 94…I still struggle in certain areas of comprehension. I received a score of 1070 on the SAT, (540 Reading & 530 Math), and am barely scraping by in my college algebra class. Honestly, I would be ashamed if any of my coworkers knew I barely could do high school-level algebra.

This person thinks they’re reinforcing their point by listing two different tests, but actually a 1070 on the SAT corresponds to about 104, a full ten points higher. Based on other things in their post – their correct use of big words and complicated sentence structure, their mention that they work a successful job in cybersecurity, the fact that they read a philosophy/​psychology subreddit for fun – I’m guessing the 104 is closer to the truth.

From the comments on the same Reddit thread:

Interesting, I hope more people who have an avg. or low IQ post. Personally I had an IQ of 90 or so, but the day of the test I stayed up almost the entire night, slept maybe two hours and as a naive caffeine user I had around 500 mg caffeine. Maybe low IQ people do that.
I did IQTest.dk Raven’s test on impulse after seeing a video of Peterson’s regarding the importance of IQ, not in a very focused mode, almost ADHD like with rumination and I scored 108, but many claim low scores by around 0.5-1 SD, so that would put me in 115-123. I also am vegan, so creatine might increase my IQ by a few points. I think I am in the 120’s, but low IQ people tend to overestimate their IQ, but at least I am certainly 108 non-verbally, which is pretty average and low.

The commenter is right that IQtest.dk usually underestimates scores compared to other tests. But even if we take it at face value, his first score was almost twenty points off. By the official numbers, that should only happen once in every 15,000 people. In reality, someone posts a thread about it on Reddit and another person immediately shows up to say “Yeah, that happened to me”.

Nobel-winning physicist Richard Feynman famously scored “only” 124 on an IQ test in school – still bright, but nowhere near what you would expect of a Nobelist. Some people point out that it might have been biased towards measuring verbal rather than math abilities – then again, Feynman’s autobiography (admittedly edited and stitched together by a ghostwriter) sold 500,000 copies and made the New York Times bestseller list. So either his tested IQ was off by at least 30 points (supposed chance of this happening: 1⁄505 million), or IQ isn’t real and all of the studies showing that it is are made up by lizardmen to confuse us. In either case, you should be less concerned if your own school IQ tests seem kind of low.

I don’t know why there’s such a discrepancy between the official reliability numbers and the ones that anecdotally make sense. My guess is that the official studies give the tests better somehow. They use professional test administrators instead of overworked school counselors. They give them at a specific time of day instead of while the testee is half-asleep. They don’t let people take a bunch of caffeine before the test. They actually write the result down in a spreadsheet they have right there instead of trusting the testee to remember it accurately.

In my own field, official studies diagnose psychiatric diseases through beautiful Structured Clinical Interviews performed to exacting guidelines. Then real doctors diagnose them through checklists that say “DO NOT USE FOR DIAGNOSIS” in big letters on the top. If psychometrics is at all similar, the clashing numbers aren’t much of a mystery.

But two other points that might also be involved.

First, on a population level IQ is very stable with age. Over a study of 87,498 Scottish children, age 11 IQ and adult IQ correlated at 0.66, about as strong and impressive a correlation as you’ll ever find in the social sciences. But “correlation of 0.66” is also known as “only predicts 44% of the variance”. On an individual level, it is totally possible and not even that surprising to have an IQ of 100 at age 11 but 120 at age 30, or vice versa. Any IQ score you got before high school should be considered a plausible prediction about your adult IQ and nothing more.

Second, the people who get low IQ scores, are shocked, find their whole world tumbling in on themselves, and desperately try to hold on to their dream of being an intellectual – are not a representative sample of the people who get low IQ scores. The average person who gets a low IQ score says “Yup, guess that would explain why I’m failing all my classes”, and then goes back to beating up nerds. When you see someone saying “Help, I got a low IQ score, I’ve double-checked the standard deviation of all of my subscores and found some slight discrepancy but I’m not sure if that counts as Bayesian evidence that the global value is erroneous”, then, well – look, I wouldn’t be making fun of these people if I didn’t constantly come across them. You know who you are.

Just for fun, I analyzed the lowest IQ scores in my collection of SSC/​LW surveys. I was only able to find three people who claimed to have an IQ ≤ 100 plus gave SAT results. All three had SAT scores corresponding to IQs in the 120s.

I conclude that at least among the kind of people I encounter and who tend to send me these emails, IQ estimates are pretty terrible.

This is absolutely consistent with population averages of thousands of IQ estimates still being valuable and useful research tools. It just means you shouldn’t use it on yourself. Statistics is what tells us that almost everybody feels stimulated on amphetamines. Reality is my patient who consistently goes to sleep every time she takes Adderall. Neither the statistics nor the lived experience are wrong – but if you use one when you need the other, you’re going to have a bad time.

III.

The second problem is that even if you avoid the problems mentioned above and measure IQ 100% correctly, it’s just not that usefully predictive.

Isn’t that heresy?! Isn’t IQ the most predictive thing we have? Doesn’t it affect every life outcome as proven again and again in well-replicated experiments?

Yes! I’m not denying any of that. I’m saying that things that are statistically true aren’t always true for any individual.

Once again, consider the analogy to family transmission of income. Your parents’ socioeconomic status correlates with your own at about r = 0.2 to 0.3, depending on how you define “socioeconomic status”. By coincidence, this is pretty much the same correlation that Strenze (2006) found for IQ and socioeconomic status. Everyone knows that having rich parents is pretty useful if you want to succeed. But everyone also knows that rich parents aren’t the only thing that goes into success. Someone from a poor family who tries really hard and gets a lot of other advantages still has a chance to make it. A sociologist or economist should be very interested in parent-child success correlations; the average person trying to get ahead should just shrug, realize things are going to be a little easier/​harder than they would have been otherwise, and get on with their life.

And this isn’t just about gaining success by becoming an athlete or musician or some other less-intellectual pursuit. Chess talent is correlated with IQ at 0.24, about the same as income. IQ is some complicated central phenomenon that contributes a little to every cognitive skill, but it doesn’t entirely determine any cognitive skill. It’s not just that you can have an average IQ and still be a great chess player if you work hard enough – that’s true, but it’s not just that. It’s that you can have an average IQ and still have high levels of innate talent in chess. It’s not quite as likely as if you have a high IQ, but it’s very much in the range of possibility. And then you add in the effects of working hard enough, and then you’re getting somewhere.

Here is a table of professions by IQ, a couple of decades out of date but probably not too far off (cf. discussion here):

I don’t know how better to demonstrate this idea of “statistically solid, individually shaky”. On a population level, we see that the average doctor is 30 IQ points higher than the average janitor, that college professors are overwhelmingly high-IQ, and we think yeah, this is about what we would hope for from a statistic measuring intelligence. But on an individual level, we see that below-average IQ people sometimes become scientists, professors, engineers, and almost anything else you could hope for.

IV.

I’m kind of annoyed I have to write this post. After investing so much work debunking IQ denialists, I feel like this is really – I don’t know – diluting the brand.

But I actually think it’s not as contradictory as it looks, that there’s some common thread between my posts arguing that no, IQ isn’t fake, and this one.

If you really understand the idea of a statistical predictor – if you have that gear in your brain at a fundamental level – then social science isn’t scary. You can read about IQ, or heredity, or stereotypes, or gender differences, or whatever, and you can say – ah, there’s a slight tendency for one thing to correlate with another thing. Then you can go have dinner.

If you don’t get that, then the world is terrifying. Someone’s said that IQ “correlates with” life outcomes? What the heck is “correlate with”? Did they say that only high-IQ people can be successful? That you’re doomed if you don’t get the right score on a test?

And then you can either resist that with every breath you have – deny all the data, picket the labs where it’s studied, make up silly theories about “emotional intelligence” and “grit” and what have you. Or you can surrender to the darkness, at least have the comfort of knowing that you accept the grim reality as it is.

Imagine an American who somehow gets it into his head that the Communists are about to invade with overwhelming force. He might buy a bunch of guns, turn his house into a bunker, start agitating that Communist sympathizers be imprisoned to prevent them from betraying the country when the time came. Or he might hang a red flag from his house, wear a WELCOME COMMUNIST OVERLORDS tshirt, and start learning Russian. These seem like opposite responses, but they both come from the same fundamental misconception. A lot of the culture war – on both sides – seems like this. I don’t know how to solve this except to try, again and again, to install the necessary gear and convince people that correlations are neither meaningless nor always exactly 1.0.

So please: study the science of IQ. Use IQ to explain and predict social phenomena. Work on figuring out how to raise IQ. Assume that raising IQ will have far-ranging and powerful effects on a wide variety of social problems. Just don’t expect it to predict a single person’s individual achievement with any kind of reliability. Especially not yourself.

Scott Alexander28 Sep 2017 17:12 UTC
69 points
8 comments8 min readLW link