Imagine that academic performance has a really low value for predicting job performance. [...]
Sure. It is possible to construct possible worlds in which the behaviour of the academic faculty investigated in this study is rational and unbiased and sensible and good. The question is: How credible is it that our world is one of them?
If you think it is at all credible, then I invite you to show me the numbers. Tell me what you think the actual relationship is between gender, academic performance, job performance, etc. Tell me why you think the numbers you’ve suggested are credible, and why they lead to the sort of results found in this study. Because my prediction is that to get the sort of results found in this study you will need to assume numbers that are really implausible. I could, of course, be wrong; in which case, show me. But I don’t think anything is achieved by reiterating that it’s possible for the results of this study to be consistent with good and unbiased (more precisely: “biased” only in the sense of recognizing genuine relevant correlations) decisions by the faculty. We all (I hope) know that already. “Possible” is waaaaay too low a bar.
The question is: How credible is it that our world is one of them?
Making wrong arguments isn’t good even if it leads to a true conclusion. I haven’t argued that the world happens to be shaped a certain way. I argue that your arguments are wrong. LessWrong is primarily a forum for rational debate. If you arguing for a position that I believe to be true but make arguments that are flawed I will object. That’s because arguments aren’t soldiers.
On the matter of the extend of gender discrimination I don’t have a fixed opinion. My uncertainty interval is pretty large. Not having a small uncertainty interval because you fall for flawed arguments matters. The fact that humans are by default overconfident is well replicated.
But if we become back to grades as a predictor: Google did find that academic performance is no good predictor for job performance at Google.
Google doesn’t even ask for GPA or test scores from candidates anymore, unless someone’s a year or two out of school, because they don’t correlate at all with success at the company.
Of course Google won’t give you the relevant data as an academic does, but Google is a company that wants to make money. It actually has a stake in hiring high performing individuals.
While we are at it, you argue as if scientific studies nearly always replicate. We don’t live in a world where that’s true. Political debates tend to make people overconfident.
It looks to me as if that’s because you are treating them as if they are intended to be deductive inferences when in fact they are inductive ones.
At no point have I intended to argue that (e.g.) it is impossible that the results found in this study are the result of accurate rational evaluation by the faculty in question. Only that it is very unlikely. The fact that one can construct possible worlds where their behaviour is close to optimal is of rather little relevance to that.
Google did find that academic performance is no good predictor for job performance at Google.
Among people actually hired by Google. Who (1) pretty much all have very good academic performance (see e.g. this if it’s not clear why that’s relevant) and (2) will typically have been better in other respects if worse academically, in order to get hired: see e.g. this for more information.
I conjecture that, ironically, if Google measure again, they’ll find that GPA became a better predictor of job success when they stopped using it as an important metric for selecting candidates.
you argue as if scientific studies nearly always replicate
Not intentionally. I’m aware that they don’t. None the less, scientific studies are the best we have, and it’s not like there’s a shortage of studies finding evidence of the sort of sex bias we’re discussing.
None the less, scientific studies are the best we have
“Best we have” doesn’t justify a small confidence interval. If there no good evidence available on a topic the right thing to do is to be uncertain.
it’s not like there’s a shortage of studies finding evidence of the sort of sex bias we’re discussing
The default way to act in those situations is to form your opinions based on meta-analysis.
I conjecture that, ironically, if Google measure again, they’ll find that GPA became a better predictor of job success when they stopped using it as an important metric for selecting candidates.
You basically think that a bunch of highly paid staticians make a very trivial error when a lot of money is at stake. How confident are you in that prediction?
If there is no good evidence available on a topic the right thing to do is to be uncertain.
I agree. (Did I say something to suggest otherwise?)
The default way [...] is to form your opinions based on meta-analysis.
Given the time and inclination to do the meta-analysis (or someone else who’s already done the work), yes. Have you perchance done it or read the work of someone else who has?
I agree. (Did I say something to suggest otherwise?)
On this topic it seems like your position is that you know that employers act irrationally and don’t hire woman who would perform well.
My position is that I don’t know whether or not that’s a case. That means you have a smaller confidence interval. I consider the size of that interval unjustified.
Given the time and inclination to do the meta-analysis
In the absence of that work being done it’s not good to believe that one knows the answer.
My position is that I’ve seen an awful lot of evidence, both scientific and anecdotal, that seems best explained by supposing such irrationality. A few examples:
Another study of attitudes to hiring finding that for applicants early in their career just changing the name from female to male results in dramatically more positive assessment. (The differences were smaller with a candidate several years further into his/her career.)
A famous study by Goldberg submitted identical essays under male and female names and found that it got substantially better assessments with the male name. (I should add that this one seems to have been repeated several times, sometimes getting the same result and sometimes not. Different biases at different institutions?)
In each case, of course one can come up with explanations that don’t involve bias—as some commenters in this discussion have eagerly done. But it seems to me that the evidence is well past the point where denying the existence of sexist biases is one hell of a stretch.
Sure. It is possible to construct possible worlds in which the behaviour of the academic faculty investigated in this study is rational and unbiased and sensible and good. The question is: How credible is it that our world is one of them?
If you think it is at all credible, then I invite you to show me the numbers. Tell me what you think the actual relationship is between gender, academic performance, job performance, etc. Tell me why you think the numbers you’ve suggested are credible, and why they lead to the sort of results found in this study. Because my prediction is that to get the sort of results found in this study you will need to assume numbers that are really implausible. I could, of course, be wrong; in which case, show me. But I don’t think anything is achieved by reiterating that it’s possible for the results of this study to be consistent with good and unbiased (more precisely: “biased” only in the sense of recognizing genuine relevant correlations) decisions by the faculty. We all (I hope) know that already. “Possible” is waaaaay too low a bar.
Making wrong arguments isn’t good even if it leads to a true conclusion. I haven’t argued that the world happens to be shaped a certain way. I argue that your arguments are wrong. LessWrong is primarily a forum for rational debate. If you arguing for a position that I believe to be true but make arguments that are flawed I will object. That’s because arguments aren’t soldiers.
On the matter of the extend of gender discrimination I don’t have a fixed opinion. My uncertainty interval is pretty large. Not having a small uncertainty interval because you fall for flawed arguments matters. The fact that humans are by default overconfident is well replicated.
But if we become back to grades as a predictor: Google did find that academic performance is no good predictor for job performance at Google.
Of course Google won’t give you the relevant data as an academic does, but Google is a company that wants to make money. It actually has a stake in hiring high performing individuals.
While we are at it, you argue as if scientific studies nearly always replicate. We don’t live in a world where that’s true. Political debates tend to make people overconfident.
It looks to me as if that’s because you are treating them as if they are intended to be deductive inferences when in fact they are inductive ones.
At no point have I intended to argue that (e.g.) it is impossible that the results found in this study are the result of accurate rational evaluation by the faculty in question. Only that it is very unlikely. The fact that one can construct possible worlds where their behaviour is close to optimal is of rather little relevance to that.
Among people actually hired by Google. Who (1) pretty much all have very good academic performance (see e.g. this if it’s not clear why that’s relevant) and (2) will typically have been better in other respects if worse academically, in order to get hired: see e.g. this for more information.
I conjecture that, ironically, if Google measure again, they’ll find that GPA became a better predictor of job success when they stopped using it as an important metric for selecting candidates.
Not intentionally. I’m aware that they don’t. None the less, scientific studies are the best we have, and it’s not like there’s a shortage of studies finding evidence of the sort of sex bias we’re discussing.
“Best we have” doesn’t justify a small confidence interval. If there no good evidence available on a topic the right thing to do is to be uncertain.
The default way to act in those situations is to form your opinions based on meta-analysis.
You basically think that a bunch of highly paid staticians make a very trivial error when a lot of money is at stake. How confident are you in that prediction?
I agree. (Did I say something to suggest otherwise?)
Given the time and inclination to do the meta-analysis (or someone else who’s already done the work), yes. Have you perchance done it or read the work of someone else who has?
Not very.
[EDITED to fix a punctuation typo]
On this topic it seems like your position is that you know that employers act irrationally and don’t hire woman who would perform well. My position is that I don’t know whether or not that’s a case. That means you have a smaller confidence interval. I consider the size of that interval unjustified.
In the absence of that work being done it’s not good to believe that one knows the answer.
My position is that I’ve seen an awful lot of evidence, both scientific and anecdotal, that seems best explained by supposing such irrationality. A few examples:
The study we’ve been discussing here.
A neurobiologist transitions from female to male and is immediately treated as much more competent.
Another study of attitudes to hiring finding that for applicants early in their career just changing the name from female to male results in dramatically more positive assessment. (The differences were smaller with a candidate several years further into his/her career.)
A famous study by Goldberg submitted identical essays under male and female names and found that it got substantially better assessments with the male name. (I should add that this one seems to have been repeated several times, sometimes getting the same result and sometimes not. Different biases at different institutions?)
Auditioning orchestral players behind a screen makes women do much better relative to men.
In each case, of course one can come up with explanations that don’t involve bias—as some commenters in this discussion have eagerly done. But it seems to me that the evidence is well past the point where denying the existence of sexist biases is one hell of a stretch.