If the US Census Bureau has changed its hiring practices then I may be wrong.
I am unfamiliar with how the Census Bureau hires; I was talking about the Chicago fire department, which I am fairly confident does use lotteries in its hiring and promotion decisions.
It would appear that the court and the people who wrote the law do not share your view of this particular test’s effectiveness. Perhaps you should try to convince them.
If they won’t listen to the psychometricians about g, why would I expect them to listen to me?
To clarify, the difference between my view and the court’s view is that I assume that the universally replicated finding of intelligence differences between races will show up on basically any test, because that’s what universally replicated means. Thus, unless the disparate impact is more than would be predicted by the relevant intelligence cutoff, then the burden to show disparate treatment should fall on those claiming discrimination.
The court’s view is that if there is any statistically significant difference between races (which is more strict that the previous 4/5ths rule), the burden of demonstrating differences in racial intelligence and the relevance of intelligence to the job (combined, thankfully, into one ‘validate the test for the particular job you’re hiring for’) falls on the maker of the test. But this falls on the maker of every test, making testing much more costly (and thus much less used) than it has it be, with the resulting efficiency losses. If you would like to use an extensively researched and validated IQ test for your narrow position (perhaps only one person will have this job at your company), that’s not possible- you have to pay for experts to design a test for every position you would like to use a test for and validate that it works for that position, despite copious research demonstrating that a test that targets g specifically will be comparably effective to a specifically-designed test that targets performance on that job.
So you claim these courts (and lawmakers) all know this research on g, and you can’t imagine any better way to present it?
Anyway, you said:
Take recent firefighting anti-discrimination court cases as an example. The legally approved way to conduct promotion testing is to pass over 90% of the people, and then randomly select from everyone who passed. The legally disapproved way is to test everyone, keep the scores as numbers, sort them, and promote from the top of the list going down.
This is false. The first is almost exactly what the Chicago fire department got slapped for doing, and the courts likewise said it would illegal for the NY department. The second is what the US Census Bureau did, and appears perfectly legal due to their test intuitively matching the jobs. This makes no mention of it, instead attacking the Bureau’s use of a binary cut-off.
The court’s explicit motive explains all this quite well. For pointing this out I lost around 50 karma.
So you claim these courts (and lawmakers) all know this research on g, and you can’t imagine any better way to present it?
I don’t know what they know or don’t know, and it’s not clear to me that the presentation rather than the content of the research is the issue.
The first is almost exactly what the Chicago fire department got slapped for doing
All of the discrimination lawsuits I’ve seen for the Chicago fire department, the courts have decided in favor of the city, but I doubt I’ve seen all of them. Which case are you thinking of?
For pointing this out I lost around 50 karma.
I can’t comment as to why others downvoted you; I did not. The primary thing I’ve noticed in discussing this issue with you is that you have several times declared a collection of claims false, which I would replace with putting forth specific contrasting claims. If you want to argue that promoting by lottery, after getting rid of some portion of the applicant pool by using a test, is legally disapproved, then make just that argument, and then we would discuss just that issue instead of having to figure out which issue we’re discussing. If you want to argue that the burden of proof should be on the employer to validate any test which has different score or pass distributions for different groups, then say that clearly, and so on.
In previous research I found this one, brought by white firefighters protesting the affirmative action policies in Chicago, and while I recall a second I’m on a different computer and so can’t easily check my history.
But I don’t think that case makes the point you want it to make. It does not disapprove of hiring by lottery- indeed, the remedy involves selecting which African Americans (but not white or other races!) who scored between 65 and 88 (who are still interested) will get the available jobs by lottery- they just think that the city did not put the passing score bar low enough, and the standard they used to determine what was “low enough” was the disparate impact standard, not any sort of job performance criterion.
[Edit]: I should clarify that, again, the court’s decision is made with the presumption that tests are guilty until proven innocent, and so when the decision says “the test was biased” or “there was no evidence that the test was necessary,” they do not mean that “there is evidence that the test was biased” or “there was evidence that the test was not necessary,” they just mean “there was not sufficient presented evidence that the test was necessary.”
I think I’ve disproven the factual basis you gave for your speculation: the real standard is orthogonal to your cutoff-with-lottery versus rank-by-test-scores.
And it’s now 80 karma paperclips. Do you know how ridiculous this looks, how badly Less Wrong is breaking its own rules of conversation?
the real standard is orthogonal to your cutoff-with-lottery versus rank-by-test-scores.
The real standard was that ‘disparate impact’ of the black pass rate being less than 80% of the white pass rate was prima facie evidence of discrimination; the more recent cases suggest that any statistically significant difference can be evidence of discrimination. If you ever got the impression that I didn’t think that was how the courts behaved, I apologize for the miscommunication on my end. (I left out the four-fifths part, and just mentioned ‘over 90%’, because I thought it would be more communicative than adding the additional detail.)
I still maintain that if you are seeking to promote, say, 5% of the population, no merit-based test which gives you the top 5% of the population will pass the four-fifths rule in the presence of underlying racial differences in merit. (This would be the ‘rank-by-test-scores’ approach.) Promoting from the entire pool at random would not discriminate by race, but it also wouldn’t discriminate by merit. The way to both have some merit selection, and not run afoul of the four-fifths rule, is to set some cutoff such that the rate at which blacks are above the cutoff is at least 80% of the rate at which whites are above the cutoff, declare everyone above that cutoff as having passed, and then promote randomly from those who passed.
It’s still not clear to me what you think you’ve disproven, or why you think you’ve disproven it. How long this conversation has gone and the propensity for others to downvote your comments suggest to me that it may be wise to call this conversation done here, or move it to PMs if you’re interested in carrying on.
I’ve lost about 150 karma (and the actual loss if it wasn’t for people voting up −1 comments would probably be more like 250). The moderators have done squat, if we even have anything that passes for moderators. (The admins, then.)
For what? Is there a particular explicit rule that’s being broken?
It violates “How should I use my voting powers?” in http://wiki.lesswrong.com/wiki/FAQ#Site_Etiquette_and_Social_Norms but even aside from that, most forums have a rule of “if you’re enough of a dick, we can ban you regardless of whether there’s an explicit rule prohibiting your exact behavior”.
But this falls on the maker of every test, making testing much more costly (and thus much less used) than it has it be
So every job I’ve ever applied for required tests, and all of them looked more like general intelligence tests than specific (the standard brain teasers about buckets of water, geometry questions,etc all for statistical programming jobs). With the exception of one insurance company (who disguised their geometry questions as programming questions), none of these companies tried to pretend these were directly applicable to job performance. To my knowledge, none of these companies have been sued.
If anything, my experience is that testing is overused. A recent hire I wanted (who I’ve worked with before, and who is very competent at exactly what we need) was refused on the basis poor performance on two tests. I’ve consulted for several companies that have expressed that they hired me as a consultant because their HR’s testing procedures have made staffing too inflexible.
I’m fairly confident that you’d have an easier time in court of proving the relevance of g (or proxies for it) to statistical programming than to, say, firefighting.
So every job I’ve ever applied for required tests, and all of them looked more like general intelligence tests than specific (the standard brain teasers about buckets of water, geometry questions,etc all for statistical programming jobs).
So, a handful of brain teasers issued and interpreted by non-experts is surely inferior to an IQ test. So why don’t we have nationally recognized agencies that administer IQ tests, that they then report to potential employers at your request, like the SAT and colleges?
(And it is unfortunate about that hire- organizations should make the most of local knowledge like that, but often fail to. Hiring people as consultants might be more efficient, though, especially if you know the person has the skills for the job you need done now but might not have the skills for the next job you need.)
I am unfamiliar with how the Census Bureau hires; I was talking about the Chicago fire department, which I am fairly confident does use lotteries in its hiring and promotion decisions.
If they won’t listen to the psychometricians about g, why would I expect them to listen to me?
To clarify, the difference between my view and the court’s view is that I assume that the universally replicated finding of intelligence differences between races will show up on basically any test, because that’s what universally replicated means. Thus, unless the disparate impact is more than would be predicted by the relevant intelligence cutoff, then the burden to show disparate treatment should fall on those claiming discrimination.
The court’s view is that if there is any statistically significant difference between races (which is more strict that the previous 4/5ths rule), the burden of demonstrating differences in racial intelligence and the relevance of intelligence to the job (combined, thankfully, into one ‘validate the test for the particular job you’re hiring for’) falls on the maker of the test. But this falls on the maker of every test, making testing much more costly (and thus much less used) than it has it be, with the resulting efficiency losses. If you would like to use an extensively researched and validated IQ test for your narrow position (perhaps only one person will have this job at your company), that’s not possible- you have to pay for experts to design a test for every position you would like to use a test for and validate that it works for that position, despite copious research demonstrating that a test that targets g specifically will be comparably effective to a specifically-designed test that targets performance on that job.
So you claim these courts (and lawmakers) all know this research on g, and you can’t imagine any better way to present it?
Anyway, you said:
This is false. The first is almost exactly what the Chicago fire department got slapped for doing, and the courts likewise said it would illegal for the NY department. The second is what the US Census Bureau did, and appears perfectly legal due to their test intuitively matching the jobs. This makes no mention of it, instead attacking the Bureau’s use of a binary cut-off.
The court’s explicit motive explains all this quite well. For pointing this out I lost around 50 karma.
I don’t know what they know or don’t know, and it’s not clear to me that the presentation rather than the content of the research is the issue.
All of the discrimination lawsuits I’ve seen for the Chicago fire department, the courts have decided in favor of the city, but I doubt I’ve seen all of them. Which case are you thinking of?
I can’t comment as to why others downvoted you; I did not. The primary thing I’ve noticed in discussing this issue with you is that you have several times declared a collection of claims false, which I would replace with putting forth specific contrasting claims. If you want to argue that promoting by lottery, after getting rid of some portion of the applicant pool by using a test, is legally disapproved, then make just that argument, and then we would discuss just that issue instead of having to figure out which issue we’re discussing. If you want to argue that the burden of proof should be on the employer to validate any test which has different score or pass distributions for different groups, then say that clearly, and so on.
What?
In previous research I found this one, brought by white firefighters protesting the affirmative action policies in Chicago, and while I recall a second I’m on a different computer and so can’t easily check my history.
But I don’t think that case makes the point you want it to make. It does not disapprove of hiring by lottery- indeed, the remedy involves selecting which African Americans (but not white or other races!) who scored between 65 and 88 (who are still interested) will get the available jobs by lottery- they just think that the city did not put the passing score bar low enough, and the standard they used to determine what was “low enough” was the disparate impact standard, not any sort of job performance criterion.
[Edit]: I should clarify that, again, the court’s decision is made with the presumption that tests are guilty until proven innocent, and so when the decision says “the test was biased” or “there was no evidence that the test was necessary,” they do not mean that “there is evidence that the test was biased” or “there was evidence that the test was not necessary,” they just mean “there was not sufficient presented evidence that the test was necessary.”
I think I’ve disproven the factual basis you gave for your speculation: the real standard is orthogonal to your cutoff-with-lottery versus rank-by-test-scores.
And it’s now 80 karma paperclips. Do you know how ridiculous this looks, how badly Less Wrong is breaking its own rules of conversation?
The real standard was that ‘disparate impact’ of the black pass rate being less than 80% of the white pass rate was prima facie evidence of discrimination; the more recent cases suggest that any statistically significant difference can be evidence of discrimination. If you ever got the impression that I didn’t think that was how the courts behaved, I apologize for the miscommunication on my end. (I left out the four-fifths part, and just mentioned ‘over 90%’, because I thought it would be more communicative than adding the additional detail.)
I still maintain that if you are seeking to promote, say, 5% of the population, no merit-based test which gives you the top 5% of the population will pass the four-fifths rule in the presence of underlying racial differences in merit. (This would be the ‘rank-by-test-scores’ approach.) Promoting from the entire pool at random would not discriminate by race, but it also wouldn’t discriminate by merit. The way to both have some merit selection, and not run afoul of the four-fifths rule, is to set some cutoff such that the rate at which blacks are above the cutoff is at least 80% of the rate at which whites are above the cutoff, declare everyone above that cutoff as having passed, and then promote randomly from those who passed.
It’s still not clear to me what you think you’ve disproven, or why you think you’ve disproven it. How long this conversation has gone and the propensity for others to downvote your comments suggest to me that it may be wise to call this conversation done here, or move it to PMs if you’re interested in carrying on.
I’ve lost about 150 karma (and the actual loss if it wasn’t for people voting up −1 comments would probably be more like 250). The moderators have done squat, if we even have anything that passes for moderators. (The admins, then.)
The moldbuggians seem to prefer downvoting to argument. Maybe it’s cooler.
What exactly do you think the moderators should do?
They could ban the stalker.
Alternately, they could release the stalker’s name.
And of course they could always use the incident as evidence that the code needs support for other measures.
For what? Is there a particular explicit rule that’s being broken?
Yes, I know how well-kept gardens die. But that’s not the only way for a garden to die.
It violates “How should I use my voting powers?” in http://wiki.lesswrong.com/wiki/FAQ#Site_Etiquette_and_Social_Norms but even aside from that, most forums have a rule of “if you’re enough of a dick, we can ban you regardless of whether there’s an explicit rule prohibiting your exact behavior”.
So every job I’ve ever applied for required tests, and all of them looked more like general intelligence tests than specific (the standard brain teasers about buckets of water, geometry questions,etc all for statistical programming jobs). With the exception of one insurance company (who disguised their geometry questions as programming questions), none of these companies tried to pretend these were directly applicable to job performance. To my knowledge, none of these companies have been sued.
If anything, my experience is that testing is overused. A recent hire I wanted (who I’ve worked with before, and who is very competent at exactly what we need) was refused on the basis poor performance on two tests. I’ve consulted for several companies that have expressed that they hired me as a consultant because their HR’s testing procedures have made staffing too inflexible.
I’m fairly confident that you’d have an easier time in court of proving the relevance of g (or proxies for it) to statistical programming than to, say, firefighting.
So, a handful of brain teasers issued and interpreted by non-experts is surely inferior to an IQ test. So why don’t we have nationally recognized agencies that administer IQ tests, that they then report to potential employers at your request, like the SAT and colleges?
(And it is unfortunate about that hire- organizations should make the most of local knowledge like that, but often fail to. Hiring people as consultants might be more efficient, though, especially if you know the person has the skills for the job you need done now but might not have the skills for the next job you need.)