One possible source of information here is what goes in other countries that do use IQ tests for this end.
In England I’ve witnessed de facto IQ tests being used for grammar schooladmissions, and job applications to banks & auditors. So some English headmasters & professional service companies’ HR departments think they’re useful, which does not prove they are useful, but is good evidence of it.
I suspect the tests are more useful for the schools than the graduate employers, since there’s less scope for cheating with the school tests (which are virtually always administered like exams AFAIK) than the employment tests (the first round of which can sometimes be taken on any Internet-connected computer, such as the one you’re sitting at in a campus café while your mate hangs over the back of your seat and has a jolly time alternating between calling out answers and ribbing you for feeling the need to cheat), and the employers are sampling a more range-restricted population (stereotypically, 21- or 22-year-olds with 2:1 or 1st class quantitative degrees from impressive-sounding universities).
I don’t doubt that IQ tests can be useful, but I’m sceptical that one shorter than 10 minutes can be so robust as to make anything else (save a test for willingness to submission) unnecessary.
I was speaking more to your Campbelling point than JM’s shortness point. I’ll quickly address the latter now.
I’ve seen papers mention developing or using short versions of IQ tests that give scores correlating well (r > 0.8) with scores on the original full tests. I don’t know whether these “short” tests last <10 minutes, but it wouldn’t surprise me. Also, the US General Social Survey has a 10-item (I think?) vocabulary test that works as a passable proxy for IQ, even though it must take only 5 minutes or so to administer. Given these two existing kinds of test, I wouldn’t be surprised if a 9-minute IQ test had enough reliability to be almost as good as a full-length IQ test.
Note that JM wasn’t saying his two suggested test types would make other tests unnecessary. I agree with you that short tests for IQ & submissiveness wouldn’t make all other predictors of job performance superfluous. (Conscientiousness tests & structured interviews, if I remember rightly, both predict job performance even after controlling for IQ.) JM just said they’d be better predictors than anything else, which wouldn’t surprise me much. AFAIK, a full-length IQ test is the best general predictor of job performance, and the same would presumably be true for a 9-minute IQ test were it reliable enough (q.v. the previous paragraph).
[Edit: Christ, I really like that “wouldn’t surprise me” phrase, don’t I?]
Also, the US General Social Survey has a 10-item (I think?) vocabulary test that works as a passable proxy for IQ, even though it must take only 5 minutes or so to administer.
That sounds even more amenable to that problem (if the items are the same in all editions of the test): if employers started to use such a test to sort prospective employees, within a year applicants would probably start memorizing those particular 10 words and >80% of them would ace the test.
From what I’ve seen in certain similar situations, some people would prepare for such a test by trying to memorize every single word in the wordbank, even if there are several thousands of them; OTOH, if the bank is large enough, then the extent to which they’d manage to do that would probably correlate with IQ strongly enough. (Unless some candidates use SRSs and other don’t.)
People have already pointed out that IQ tests are considered to be illegal discrimination, and have been ignoring that, but I think a good case can be made that it is proper to treat them as illegal discrimination. Imagine that a test genuinely detects ability to perform at the job but is also correlated with race independently of whether people of that race can do the job. Under those circumstances, if you use an IQ test to test employees, you’ll 1) avoid stupid employees and 2) avoid employees of one race in favor of equally competent people of another.
The overall result will be beneficial to you, since avoiding stupid employees benefits you while avoiding employees of one race has neutral effects on you. But the overall effect on people of that race will be devastating.
This will never be corrected by the market. The employers, after all, benefit unless there are so few available employees that cutting out the good ones of the wrong race reduces the pool of potential employees enough to affect the overall employee quality to a significant degree. How could we prevent this other than by banning IQ tests?
The overall result will be beneficial to you, since avoiding stupid employees benefits you while avoiding employees of one race has neutral effects on you. But the overall effect on people of that race will be devastating.
This ‘devastating’ effect sounds drastic. It also seems to directly require the assumption that the performance of people of the particular race perform devastatingly poor on IQ tests. That’s a rather dubious assumption and when asked to ‘imagine’ a world where this idea is so, any conclusions we draw only apply in that imaginary universe, not this one. This is not a ‘good case’.
When arguing that IQ tests should be illegal discrimination due to drastic racial bias of IQ tests the reasoning from the premise to conclusion is straightforward. The bulk of a ‘good case’ must be in providing evidence for the premise. Maybe there is sufficient evidence to make that claim about IQ tests but imagining it is not sufficient.
This ‘devastating’ effect sounds drastic. It also seems to directly require the assumption that the performance of people of the particular race perform devastatingly poor on IQ tests.
No, it doesn’t. It only requires the assumption that 1) such people score slightly more poorly, and 2) employers begin choosing employees from the tail end of the distribution. A slight difference in scores on the test will reduce the percentage at the tail end out of proportion to the size of the difference. I don’t remember the formal name for this, but it’s very well known.
Edit: Also, consider what percentage of the population needs to be unemployed for the problem to be considered serious. 10% is bad. If test bias causes an additional 5 or 10% to be unemployed, that’s going to be a huge effect even if 5% or 10% is only a small part of the entire population. Nobody says that the unemployment rate needs to get up to 25% or 50% in order for unemployment to be a serious problem.
No, it doesn’t. It only requires the assumption that 1) such people score slightly more poorly, and 2) employers begin choosing employees from the tail end of the distribution. A slight difference in scores on the test will reduce the percentage at the tail end out of proportion to the size of the difference. I don’t remember the formal name for this, but it’s very well known.
I will not quibble about what constitutes ‘devastatingly poor’ beyond observing that I of course agree that the difference need not be numerically large in order to be devastating. I will again point out that you are assuming that there is sufficient bias in the IQ test that if used (instead of whatever other form of measurement or selection could be adopted) the change will be devastating. That is a significant claim, particularly when it is quite possible that subjective evaluations of intelligence by interviewers are more biased than IQ tests. I expect IQ may be somewhat racially biased. I know for certain that unstructured human arbitration is racially biased. I don’t know for sure to which is worse but do observe that it is sufficiently controversial that some evidence is required.
You asked the reader to imagine a world where some assumption holds—which is fine as far as it goes. It becomes an error in reasoning when you jump from “imagine...” to “a good case” without providing evidence that the imaginary scenario applies to reality. Even the bare statement “I assert that IQ tests are more racially biased than whatever is used in their stead” would have made the case at least coherent, albeit still weak.
The argument was in response to the implicit assumption that even if the tests are biased, we shouldn’t worry about that as long as they can predict performance. “Imagine” takes the place of the word “if” in there and the intended conclusion is “yes, you should worry about whether the tests are biased, because if they are biased, that would be bad.” To make that conclusion I do not need to provide evidence that the imaginary scenario applies to reality.
If I say “imagine that you jumped off a cliff. You’ll get smashed. Maybe you should avoid jumping off cliffs”, I don’t need to provide evidence that you jump off cliffs, because the conclusion is in the form of a conditional that already conditions on whether the imaginary scenario is real.
One possible source of information here is what goes in other countries that do use IQ tests for this end.
In England I’ve witnessed de facto IQ tests being used for grammar school admissions, and job applications to banks & auditors. So some English headmasters & professional service companies’ HR departments think they’re useful, which does not prove they are useful, but is good evidence of it.
I suspect the tests are more useful for the schools than the graduate employers, since there’s less scope for cheating with the school tests (which are virtually always administered like exams AFAIK) than the employment tests (the first round of which can sometimes be taken on any Internet-connected computer, such as the one you’re sitting at in a campus café while your mate hangs over the back of your seat and has a jolly time alternating between calling out answers and ribbing you for feeling the need to cheat), and the employers are sampling a more range-restricted population (stereotypically, 21- or 22-year-olds with 2:1 or 1st class quantitative degrees from impressive-sounding universities).
I don’t doubt that IQ tests can be useful, but I’m sceptical that one shorter than 10 minutes can be so robust as to make anything else (save a test for willingness to submission) unnecessary.
I was speaking more to your Campbelling point than JM’s shortness point. I’ll quickly address the latter now.
I’ve seen papers mention developing or using short versions of IQ tests that give scores correlating well (r > 0.8) with scores on the original full tests. I don’t know whether these “short” tests last <10 minutes, but it wouldn’t surprise me. Also, the US General Social Survey has a 10-item (I think?) vocabulary test that works as a passable proxy for IQ, even though it must take only 5 minutes or so to administer. Given these two existing kinds of test, I wouldn’t be surprised if a 9-minute IQ test had enough reliability to be almost as good as a full-length IQ test.
Note that JM wasn’t saying his two suggested test types would make other tests unnecessary. I agree with you that short tests for IQ & submissiveness wouldn’t make all other predictors of job performance superfluous. (Conscientiousness tests & structured interviews, if I remember rightly, both predict job performance even after controlling for IQ.) JM just said they’d be better predictors than anything else, which wouldn’t surprise me much. AFAIK, a full-length IQ test is the best general predictor of job performance, and the same would presumably be true for a 9-minute IQ test were it reliable enough (q.v. the previous paragraph).
[Edit: Christ, I really like that “wouldn’t surprise me” phrase, don’t I?]
That sounds even more amenable to that problem (if the items are the same in all editions of the test): if employers started to use such a test to sort prospective employees, within a year applicants would probably start memorizing those particular 10 words and >80% of them would ace the test.
The words are not always the same. They are selected from a standard wordbank.
From what I’ve seen in certain similar situations, some people would prepare for such a test by trying to memorize every single word in the wordbank, even if there are several thousands of them; OTOH, if the bank is large enough, then the extent to which they’d manage to do that would probably correlate with IQ strongly enough. (Unless some candidates use SRSs and other don’t.)
People have already pointed out that IQ tests are considered to be illegal discrimination, and have been ignoring that, but I think a good case can be made that it is proper to treat them as illegal discrimination. Imagine that a test genuinely detects ability to perform at the job but is also correlated with race independently of whether people of that race can do the job. Under those circumstances, if you use an IQ test to test employees, you’ll 1) avoid stupid employees and 2) avoid employees of one race in favor of equally competent people of another.
The overall result will be beneficial to you, since avoiding stupid employees benefits you while avoiding employees of one race has neutral effects on you. But the overall effect on people of that race will be devastating.
This will never be corrected by the market. The employers, after all, benefit unless there are so few available employees that cutting out the good ones of the wrong race reduces the pool of potential employees enough to affect the overall employee quality to a significant degree. How could we prevent this other than by banning IQ tests?
This ‘devastating’ effect sounds drastic. It also seems to directly require the assumption that the performance of people of the particular race perform devastatingly poor on IQ tests. That’s a rather dubious assumption and when asked to ‘imagine’ a world where this idea is so, any conclusions we draw only apply in that imaginary universe, not this one. This is not a ‘good case’.
When arguing that IQ tests should be illegal discrimination due to drastic racial bias of IQ tests the reasoning from the premise to conclusion is straightforward. The bulk of a ‘good case’ must be in providing evidence for the premise. Maybe there is sufficient evidence to make that claim about IQ tests but imagining it is not sufficient.
No, it doesn’t. It only requires the assumption that 1) such people score slightly more poorly, and 2) employers begin choosing employees from the tail end of the distribution. A slight difference in scores on the test will reduce the percentage at the tail end out of proportion to the size of the difference. I don’t remember the formal name for this, but it’s very well known.
Edit: Also, consider what percentage of the population needs to be unemployed for the problem to be considered serious. 10% is bad. If test bias causes an additional 5 or 10% to be unemployed, that’s going to be a huge effect even if 5% or 10% is only a small part of the entire population. Nobody says that the unemployment rate needs to get up to 25% or 50% in order for unemployment to be a serious problem.
I will not quibble about what constitutes ‘devastatingly poor’ beyond observing that I of course agree that the difference need not be numerically large in order to be devastating. I will again point out that you are assuming that there is sufficient bias in the IQ test that if used (instead of whatever other form of measurement or selection could be adopted) the change will be devastating. That is a significant claim, particularly when it is quite possible that subjective evaluations of intelligence by interviewers are more biased than IQ tests. I expect IQ may be somewhat racially biased. I know for certain that unstructured human arbitration is racially biased. I don’t know for sure to which is worse but do observe that it is sufficiently controversial that some evidence is required.
You asked the reader to imagine a world where some assumption holds—which is fine as far as it goes. It becomes an error in reasoning when you jump from “imagine...” to “a good case” without providing evidence that the imaginary scenario applies to reality. Even the bare statement “I assert that IQ tests are more racially biased than whatever is used in their stead” would have made the case at least coherent, albeit still weak.
The argument was in response to the implicit assumption that even if the tests are biased, we shouldn’t worry about that as long as they can predict performance. “Imagine” takes the place of the word “if” in there and the intended conclusion is “yes, you should worry about whether the tests are biased, because if they are biased, that would be bad.” To make that conclusion I do not need to provide evidence that the imaginary scenario applies to reality.
If I say “imagine that you jumped off a cliff. You’ll get smashed. Maybe you should avoid jumping off cliffs”, I don’t need to provide evidence that you jump off cliffs, because the conclusion is in the form of a conditional that already conditions on whether the imaginary scenario is real.