The other is that my observation has been that the most mathematically talented people who I know have usually have scored 800 on the math SAT … You seem to be claiming that a ceiling effect) makes the test a bad measurement instrument, which certainly is true to some extent, and which is a priori plausible, but you may have been less lucky than you think.
Specifically, a bad measurement instrument at differentiating very high levels of mathematical ability. It works as well as you would expect when the measurement error doesn’t hit the ceiling or floor.
I should be clearer about my ‘luck’ claim: what a raw score of “all but one right, one wrong” gets you depends on the percentage of students who got “all right” that year, which depends on that year’s test difficulty. Some years it’s 760, some years it’s 780, and so on. (If I remember correctly, I got both of those processed scores from taking it two times and getting the same raw score.) I do not think the underlying raw score of “all but one right, one wrong” is due to luck (in the sense of my underlying skill creates a family of rate parameters for Poisson distributions that are summed together to get a total error count, and while any sample from that distribution is stochastic the distribution is very narrow).
See my comment here. I agree to some extent, but the correlation between cognitive ability and math SAT scores is positive for all levels of cognitive ability and SAT math scores, including the highest ones (even if it becomes substantially smaller).
Added: To operationalize the situation, I would guess that the frequency with which mathematicians who have won famous prizes (Abel Prize, Fields Medal, etc.) would miss no questions at all (say, as 18 year olds) would be noticeably higher than the corresponding frequency for professors at top 50 math departments. I’ll give evidence in subsequent posts.
To operationalize the situation, I would guess that the frequency with which mathematicians who have won famous prizes (Abel Prize, Fields Medal, etc.) would miss no questions at all (say, as 18 year olds) would be noticeably higher than the corresponding frequency for professors at top 50 math departments.
I agree that I expect it would be higher, though I would describe my expectation as “modest,” which probably overlaps with “noticeable.”
Specifically, a bad measurement instrument at differentiating very high levels of mathematical ability. It works as well as you would expect when the measurement error doesn’t hit the ceiling or floor.
I should be clearer about my ‘luck’ claim: what a raw score of “all but one right, one wrong” gets you depends on the percentage of students who got “all right” that year, which depends on that year’s test difficulty. Some years it’s 760, some years it’s 780, and so on. (If I remember correctly, I got both of those processed scores from taking it two times and getting the same raw score.) I do not think the underlying raw score of “all but one right, one wrong” is due to luck (in the sense of my underlying skill creates a family of rate parameters for Poisson distributions that are summed together to get a total error count, and while any sample from that distribution is stochastic the distribution is very narrow).
See my comment here. I agree to some extent, but the correlation between cognitive ability and math SAT scores is positive for all levels of cognitive ability and SAT math scores, including the highest ones (even if it becomes substantially smaller).
Added: To operationalize the situation, I would guess that the frequency with which mathematicians who have won famous prizes (Abel Prize, Fields Medal, etc.) would miss no questions at all (say, as 18 year olds) would be noticeably higher than the corresponding frequency for professors at top 50 math departments. I’ll give evidence in subsequent posts.
I agree that I expect it would be higher, though I would describe my expectation as “modest,” which probably overlaps with “noticeable.”