SAT is very g-loaded, so it would be susceptible to the same practice effect that IQ tests in general are susceptible to. If you look at SAT/IQ tables, the 20-40 point increase Tyler cites corresponds to about 1.5-3 IQ points. This is consistent with the typical magnitude (< 5 points) of the practice effects on IQ test scores. Your “hundreds of points” are wildly inconsistent with this. The only way I could see that happen is if quite a bit of the SAT would test for skills that can be practiced but don’t correlate with g. Not very likely.
The way to reconcile your experience with the evidence is to note that the score on a low-stakes practice test is just not comparable to the score on the real thing (with or without test prep). It’s not that implausible to believe that, say, 10% of people (more than enough to account for your anecdotes) will score at least 100 points less on an early practice test than they could score on the real thing at the same level of preparation. It’s hard to trick your brain into believing that something is high-stakes when it isn’t.
ETA: On reflection, the low-stakes hypothesis probably doesn’t account for too much of the puzzle. In particular, it doesn’t explain any gain between consecutive low-stakes practice tests. I think James Miller’s explanation takes the cake. The SAT-g correlation is likely a lot lower for a population not proficient in English.
The only way I could see that happen is if quite a bit of the SAT would test for skills that can be practiced but don’t correlate with g. Not very likely.
Not likely?? It’s certain!
If you know the scoring rules and their implications like when to guess and when to leave it blank, that can get you points you might miss from leaving it blank and reduce your penalty on things you’d have gotten wrong.
If you know better how to manage your time, then you won’t end up rushed.
Simply having done it before reduces the stress of the situation and can enable better focus.
Being familiar with the style of questions asked will help a lot—you’ll know to expect certain odd phrasings that can trip up a naive test-taker, and in some cases you will barely need to parse, simply pattern-match. ‘Yup, this is that kind of question.’
And that’s setting aside just studying the words they’re likely to ask you about.
None of these have all that much to do with g, and I can see them producing a swing of 40 points easily, perhaps more at the lower end (you know, in the case where there are hundreds of points to gain).
This isn’t to say that intense SAT prep is a huge difference on average—it could end up inducing more freakout instead of less, it could induce someone to stay up late and not be rested, it may primarily be used by those who would already do well, it may be used as a crutch by those wouldn’t… all sorts of confounding things. But the idea that there is no significant component of the SAT that’s practicable non-g is hard to believe.
You are right that a test being g-loaded is not inconsistent with test takers receiving significant gains from repeatedly taking this test. This is why formal IQ tests used by experts are not available to the general public because practicing them can significantly raise scores and so give an inaccurate estimate of IQ.
The way to reconcile your experience with the evidence is to note that the score on a low-stakes practice test is just not comparable to the score on the real thing (with or without test prep).
This is one possibility. Thanks for bringing it up.
SAT is very g-loaded, so it would be susceptible to the same practice effect that IQ tests in general are susceptible to. If you look at SAT/IQ tables, the 20-40 point increase Tyler cites corresponds to about 1.5-3 IQ points. This is consistent with the typical magnitude (< 5 points) of the practice effects on IQ test scores. Your “hundreds of points” are wildly inconsistent with this. The only way I could see that happen is if quite a bit of the SAT would test for skills that can be practiced but don’t correlate with g. Not very likely.
The way to reconcile your experience with the evidence is to note that the score on a low-stakes practice test is just not comparable to the score on the real thing (with or without test prep). It’s not that implausible to believe that, say, 10% of people (more than enough to account for your anecdotes) will score at least 100 points less on an early practice test than they could score on the real thing at the same level of preparation. It’s hard to trick your brain into believing that something is high-stakes when it isn’t.
ETA: On reflection, the low-stakes hypothesis probably doesn’t account for too much of the puzzle. In particular, it doesn’t explain any gain between consecutive low-stakes practice tests. I think James Miller’s explanation takes the cake. The SAT-g correlation is likely a lot lower for a population not proficient in English.
Not likely?? It’s certain!
If you know the scoring rules and their implications like when to guess and when to leave it blank, that can get you points you might miss from leaving it blank and reduce your penalty on things you’d have gotten wrong.
If you know better how to manage your time, then you won’t end up rushed.
Simply having done it before reduces the stress of the situation and can enable better focus.
Being familiar with the style of questions asked will help a lot—you’ll know to expect certain odd phrasings that can trip up a naive test-taker, and in some cases you will barely need to parse, simply pattern-match. ‘Yup, this is that kind of question.’
And that’s setting aside just studying the words they’re likely to ask you about.
None of these have all that much to do with g, and I can see them producing a swing of 40 points easily, perhaps more at the lower end (you know, in the case where there are hundreds of points to gain).
This isn’t to say that intense SAT prep is a huge difference on average—it could end up inducing more freakout instead of less, it could induce someone to stay up late and not be rested, it may primarily be used by those who would already do well, it may be used as a crutch by those wouldn’t… all sorts of confounding things. But the idea that there is no significant component of the SAT that’s practicable non-g is hard to believe.
This comment is very insightful—you managed to articulate a lot of non-g factors that would explain my own observations. Thank you.
You are right that a test being g-loaded is not inconsistent with test takers receiving significant gains from repeatedly taking this test. This is why formal IQ tests used by experts are not available to the general public because practicing them can significantly raise scores and so give an inaccurate estimate of IQ.
Edited because I misread the above comment.
This is one possibility. Thanks for bringing it up.