Well, we should certainly be interested in the ability to solve new problems, especially since it looks like ‘the bit of intelligence that we don’t know how to program’, and so a way of measuring it would be a wonderful thing.
But I suspect that ‘your speed the first time you try to do the Times crossword’ is a fairly useless measure of that. If someone’s never seen anything like it before, then their score is likely to be something like 2 completed clues after 1 hour, and I’d imagine the number of completed clues is only weakly related to how good they are at that sort of thing in general, and has a large random component.
If they can solve the whole thing on their first attempt, then I’d imagine they were either an unparallelled genius or had done quite a lot of something similar before.
For the Raven’s matrices in the IQ test, your first ever score is likely to tell you a lot about how much attention you’ve previously paid to symmetries and patterns, which isn’t necessarily related to either mental speed or creativity.
It should pick out the mathematicians and artists, but it will probably also pick out people who are very interested in wallpaper.
Let’s say that what IQ tests such as RPM measured in the early 20th century is “pattern-fu”; and that pattern-fu has historically correlated with all sorts of nice things — business success, academic excellence, artistic significance, happiness in relationships, and so on. Well, that’s fine, but once you start making policy decisions — hiring, college placement, military job assignment, eugenics, etc. — on the basis of pattern-fu, Campbell’s law kicks in and weakens the correlation. You get people specializing and training pattern-fu, treating improving pattern-fu as causing nice things, without that causation actually being demonstrated.
I’d be amazed if that sort of thing wasn’t happening, but you see the same sort of effect on all sorts of things which correlate, like spelling and grammar and success in studying.
Presumably the hope with IQ tests would be that the amount of training you’d need to do to top out would be low compared to, say, learning English spelling. Allowing you to get a reliable, but not terribly costly, indicator of how good you’d become at all the other things with practice.
Well, we should certainly be interested in the ability to solve new problems, especially since it looks like ‘the bit of intelligence that we don’t know how to program’, and so a way of measuring it would be a wonderful thing.
But I suspect that ‘your speed the first time you try to do the Times crossword’ is a fairly useless measure of that. If someone’s never seen anything like it before, then their score is likely to be something like 2 completed clues after 1 hour, and I’d imagine the number of completed clues is only weakly related to how good they are at that sort of thing in general, and has a large random component.
If they can solve the whole thing on their first attempt, then I’d imagine they were either an unparallelled genius or had done quite a lot of something similar before.
For the Raven’s matrices in the IQ test, your first ever score is likely to tell you a lot about how much attention you’ve previously paid to symmetries and patterns, which isn’t necessarily related to either mental speed or creativity.
It should pick out the mathematicians and artists, but it will probably also pick out people who are very interested in wallpaper.
I wonder if this critique can go further …
Let’s say that what IQ tests such as RPM measured in the early 20th century is “pattern-fu”; and that pattern-fu has historically correlated with all sorts of nice things — business success, academic excellence, artistic significance, happiness in relationships, and so on. Well, that’s fine, but once you start making policy decisions — hiring, college placement, military job assignment, eugenics, etc. — on the basis of pattern-fu, Campbell’s law kicks in and weakens the correlation. You get people specializing and training pattern-fu, treating improving pattern-fu as causing nice things, without that causation actually being demonstrated.
I’d be amazed if that sort of thing wasn’t happening, but you see the same sort of effect on all sorts of things which correlate, like spelling and grammar and success in studying.
Presumably the hope with IQ tests would be that the amount of training you’d need to do to top out would be low compared to, say, learning English spelling. Allowing you to get a reliable, but not terribly costly, indicator of how good you’d become at all the other things with practice.