Okay, I now know that the answers to the questions I gave up with wouldn’t have occurred to me even if I had thought about them for days, except possibly one of them.
A bit the reverse for me! I’m looking at them thinking ‘how on earth did I not see that?‘, and feeling really stupid. I think if someone had said ‘composition of functions’ and ‘motion in reading order’ before the test I’d have got almost all the answers right. I think I had all the necessary tools and failed to see where to use them, which is a pretty good definition of ‘idiot’.
I bet you haven’t come into contact with much of this sort of thing before, and I further bet that if you practised doing this type of test for a while then you’d start to find them very easy.
I’d dispute the test’s claim to be ‘not culturally biased’. Obviously it doesn’t require native-speaker English or literary knowledge, but equally obviously your score will depend heavily on how much you’ve been previously exposed to ideas about symmetries and abstract mathematics.
On the other hand, it seems that you can probably learn all these ideas fairly quickly. So what your score settles down to after long practice may well be both interesting and not-culturally-biased. Some of the puzzles at the end of the test do seem to be tickling the limits of working memory.
I bet you haven’t come into contact with much of this sort of thing before, and I further bet that if you practised doing this type of test for a while then you’d start to find them very easy.
As I said, IIRC such tests are only supposed to be accurate if you hadn’t done them before.
I’d dispute the test’s claim to be ‘not culturally biased’. Obviously it doesn’t require native-speaker English or literary knowledge, but equally obviously your score will depend heavily on how much you’ve been previously exposed to ideas about symmetries and abstract mathematics.
I mentioned that before, but how would you go about designing an IQ test even less culturally biased than that?
Having read the list of recipes for constructing these matrices, this reminds me of the structure of cryptic crosswords, but done with symbols.
You’ve got the a similar list of possible tricks for the setter to use, and the same ability to combine tricks to make a harder question. You don’t have the grid pattern, whereby solving one problem gives clues to the solutions of the others, so it would be more like trying to solve a list of crossword clues without a grid.
If the analogy holds, then the ability to score on these things should be highly trainable, and your score at first would be a mix of how many you’ve done before and your ability to intuit the patterns used in construction, with a large random component, but later on it would sort of settle down to a “speed of mental Solomonoff induction”.
The classic cryptic is the one in the Times newspaper. Most people at first can only solve one or two clues, but with practice the time to solve gets lower and lower. There are people who can do it as quickly as they can write the answers in.
Speed seems to top out after a couple of years of doing them, and it wouldn’t surprise me at all if the final speed was a correlate of whatever we mean by intelligence. The other thing one might be interested in is the speed of improvement. They might be measuring different things.
Speed seems to top out after a couple of years of doing them, and it wouldn’t surprise me at all if the final speed was a correlate of whatever we mean by intelligence. The other thing one might be interested in is the speed of improvement. They might be measuring different things.
The third thing to ask about is your speed the very first time you solve one. But that’s harder to analyze, since it depends on whether you’ve tried to solve similar problems before.
Well, we should certainly be interested in the ability to solve new problems, especially since it looks like ‘the bit of intelligence that we don’t know how to program’, and so a way of measuring it would be a wonderful thing.
But I suspect that ‘your speed the first time you try to do the Times crossword’ is a fairly useless measure of that. If someone’s never seen anything like it before, then their score is likely to be something like 2 completed clues after 1 hour, and I’d imagine the number of completed clues is only weakly related to how good they are at that sort of thing in general, and has a large random component.
If they can solve the whole thing on their first attempt, then I’d imagine they were either an unparallelled genius or had done quite a lot of something similar before.
For the Raven’s matrices in the IQ test, your first ever score is likely to tell you a lot about how much attention you’ve previously paid to symmetries and patterns, which isn’t necessarily related to either mental speed or creativity.
It should pick out the mathematicians and artists, but it will probably also pick out people who are very interested in wallpaper.
Let’s say that what IQ tests such as RPM measured in the early 20th century is “pattern-fu”; and that pattern-fu has historically correlated with all sorts of nice things — business success, academic excellence, artistic significance, happiness in relationships, and so on. Well, that’s fine, but once you start making policy decisions — hiring, college placement, military job assignment, eugenics, etc. — on the basis of pattern-fu, Campbell’s law kicks in and weakens the correlation. You get people specializing and training pattern-fu, treating improving pattern-fu as causing nice things, without that causation actually being demonstrated.
I’d be amazed if that sort of thing wasn’t happening, but you see the same sort of effect on all sorts of things which correlate, like spelling and grammar and success in studying.
Presumably the hope with IQ tests would be that the amount of training you’d need to do to top out would be low compared to, say, learning English spelling. Allowing you to get a reliable, but not terribly costly, indicator of how good you’d become at all the other things with practice.
There are many online posts with solutions to each question on the test (some with better explanations). E.g., [1] and [2].
Okay, I now know that the answers to the questions I gave up with wouldn’t have occurred to me even if I had thought about them for days, except possibly one of them.
A bit the reverse for me! I’m looking at them thinking ‘how on earth did I not see that?‘, and feeling really stupid. I think if someone had said ‘composition of functions’ and ‘motion in reading order’ before the test I’d have got almost all the answers right. I think I had all the necessary tools and failed to see where to use them, which is a pretty good definition of ‘idiot’.
I bet you haven’t come into contact with much of this sort of thing before, and I further bet that if you practised doing this type of test for a while then you’d start to find them very easy.
I’d dispute the test’s claim to be ‘not culturally biased’. Obviously it doesn’t require native-speaker English or literary knowledge, but equally obviously your score will depend heavily on how much you’ve been previously exposed to ideas about symmetries and abstract mathematics.
On the other hand, it seems that you can probably learn all these ideas fairly quickly. So what your score settles down to after long practice may well be both interesting and not-culturally-biased. Some of the puzzles at the end of the test do seem to be tickling the limits of working memory.
As I said, IIRC such tests are only supposed to be accurate if you hadn’t done them before.
I mentioned that before, but how would you go about designing an IQ test even less culturally biased than that?
Sweet! Thanks Vincent, reading those pages has added 15 points to my IQ.
Having read the list of recipes for constructing these matrices, this reminds me of the structure of cryptic crosswords, but done with symbols.
You’ve got the a similar list of possible tricks for the setter to use, and the same ability to combine tricks to make a harder question. You don’t have the grid pattern, whereby solving one problem gives clues to the solutions of the others, so it would be more like trying to solve a list of crossword clues without a grid.
If the analogy holds, then the ability to score on these things should be highly trainable, and your score at first would be a mix of how many you’ve done before and your ability to intuit the patterns used in construction, with a large random component, but later on it would sort of settle down to a “speed of mental Solomonoff induction”.
The classic cryptic is the one in the Times newspaper. Most people at first can only solve one or two clues, but with practice the time to solve gets lower and lower. There are people who can do it as quickly as they can write the answers in.
Speed seems to top out after a couple of years of doing them, and it wouldn’t surprise me at all if the final speed was a correlate of whatever we mean by intelligence. The other thing one might be interested in is the speed of improvement. They might be measuring different things.
The third thing to ask about is your speed the very first time you solve one. But that’s harder to analyze, since it depends on whether you’ve tried to solve similar problems before.
Well, we should certainly be interested in the ability to solve new problems, especially since it looks like ‘the bit of intelligence that we don’t know how to program’, and so a way of measuring it would be a wonderful thing.
But I suspect that ‘your speed the first time you try to do the Times crossword’ is a fairly useless measure of that. If someone’s never seen anything like it before, then their score is likely to be something like 2 completed clues after 1 hour, and I’d imagine the number of completed clues is only weakly related to how good they are at that sort of thing in general, and has a large random component.
If they can solve the whole thing on their first attempt, then I’d imagine they were either an unparallelled genius or had done quite a lot of something similar before.
For the Raven’s matrices in the IQ test, your first ever score is likely to tell you a lot about how much attention you’ve previously paid to symmetries and patterns, which isn’t necessarily related to either mental speed or creativity.
It should pick out the mathematicians and artists, but it will probably also pick out people who are very interested in wallpaper.
I wonder if this critique can go further …
Let’s say that what IQ tests such as RPM measured in the early 20th century is “pattern-fu”; and that pattern-fu has historically correlated with all sorts of nice things — business success, academic excellence, artistic significance, happiness in relationships, and so on. Well, that’s fine, but once you start making policy decisions — hiring, college placement, military job assignment, eugenics, etc. — on the basis of pattern-fu, Campbell’s law kicks in and weakens the correlation. You get people specializing and training pattern-fu, treating improving pattern-fu as causing nice things, without that causation actually being demonstrated.
I’d be amazed if that sort of thing wasn’t happening, but you see the same sort of effect on all sorts of things which correlate, like spelling and grammar and success in studying.
Presumably the hope with IQ tests would be that the amount of training you’d need to do to top out would be low compared to, say, learning English spelling. Allowing you to get a reliable, but not terribly costly, indicator of how good you’d become at all the other things with practice.