it’s reasonable to take that as strong evidence for bias in favour of men over women that isn’t simply a proportionate response to actual differences in competence? I mean, it’s just Bayes’ theorem. How likely is that outcome if people do have such bias?
By the same logic you could say that someone who hires people with high SRT scores engages in SRT bias. Someone who hires based on SRT scores could simply reasonably believe that people with high SRT scores are more competent.
Google’s HR department has a variety of factors on which it judges candidates. A few years afterwards they reevaluate their hiring decisions. They run a regression analysis and see which factors predict job performance at Google. They learn from that analysis and switch their hiring decision to hiring people which score highly on the factors that the regression analysis found predictive.
That’s how making rational hiring decisions looks like. In the process they found that college marks aren’t very relevant for predicting job performance. Being good at Fermi estimates unfortunately isn’t as well, so those LW people who train Fermi estimates don’t get benefits anymore when they want to get a job at Google.
Given current laws Google is not allowed to put values such as gender into the mix they use to make hiring decisions. That means that Google can’t make the hiring decisions that maximize predicted job performance.
The politics of the issue also make it pretty bad PR for them to publish results about the effects of a model that includes gender if the correct value in the regression analysis would mean worse chances for woman getting a job. It’s good PR for them if the correct value would mean to favor woman. No big company that does regression analysis on job performance published data that favoring in gender would mean hiring more woman.
Factoring in gender into a regression analysis would mean that any bias against woman in subjective competence evaluations in interviews would be canceled by that factor.
Just imagine if a big company would find that by putting gender into their regression analysis they would hiring more women and get better average job performance as a result. Don’t you think those companies would lobby Washington to allow them to put gender into hiring decisions? The silence on the issue speaks.
It could be that the silencing of feminists who want to prevent “privileged” from talking about the issue is strong enough that rational companies don’t dare to speak about their need to change their hiring practices to hire more woman via making data driven arguments. If that’s the case that says a lot about the concept of privilege and it’s problem in shutting down rational arguments.
weird in that whatever it is somehow manages to make a big difference in competence without having any effect on academic performance, test scores, or reported faculty opinions
Imagine that academic performance has a really low value for predicting job performance. People that spend a lot of time preparing for tests get better academic marks. Woman spent more time than men preparing for academic tests. That means a woman of equal competence scores higher because she puts in more work. The test isn’t anymore a strict measure of competence but a measure of effort at scoring highly of the test. In that scenario it makes sense to infer that a woman with the same test score as a man is likely less competent as the man as long as you are hiring for “competence” and not for “putting in effort to game the test”.
I mean, it’s just Bayes’ theorem. How likely is that outcome if people do have such bias? How likely is it if they don’t?
If you write down the math you see that it depends on your priors for the effect size of how gender correlates with job performance.
Imagine that academic performance has a really low value for predicting job performance. [...]
Sure. It is possible to construct possible worlds in which the behaviour of the academic faculty investigated in this study is rational and unbiased and sensible and good. The question is: How credible is it that our world is one of them?
If you think it is at all credible, then I invite you to show me the numbers. Tell me what you think the actual relationship is between gender, academic performance, job performance, etc. Tell me why you think the numbers you’ve suggested are credible, and why they lead to the sort of results found in this study. Because my prediction is that to get the sort of results found in this study you will need to assume numbers that are really implausible. I could, of course, be wrong; in which case, show me. But I don’t think anything is achieved by reiterating that it’s possible for the results of this study to be consistent with good and unbiased (more precisely: “biased” only in the sense of recognizing genuine relevant correlations) decisions by the faculty. We all (I hope) know that already. “Possible” is waaaaay too low a bar.
The question is: How credible is it that our world is one of them?
Making wrong arguments isn’t good even if it leads to a true conclusion. I haven’t argued that the world happens to be shaped a certain way. I argue that your arguments are wrong. LessWrong is primarily a forum for rational debate. If you arguing for a position that I believe to be true but make arguments that are flawed I will object. That’s because arguments aren’t soldiers.
On the matter of the extend of gender discrimination I don’t have a fixed opinion. My uncertainty interval is pretty large. Not having a small uncertainty interval because you fall for flawed arguments matters. The fact that humans are by default overconfident is well replicated.
But if we become back to grades as a predictor: Google did find that academic performance is no good predictor for job performance at Google.
Google doesn’t even ask for GPA or test scores from candidates anymore, unless someone’s a year or two out of school, because they don’t correlate at all with success at the company.
Of course Google won’t give you the relevant data as an academic does, but Google is a company that wants to make money. It actually has a stake in hiring high performing individuals.
While we are at it, you argue as if scientific studies nearly always replicate. We don’t live in a world where that’s true. Political debates tend to make people overconfident.
It looks to me as if that’s because you are treating them as if they are intended to be deductive inferences when in fact they are inductive ones.
At no point have I intended to argue that (e.g.) it is impossible that the results found in this study are the result of accurate rational evaluation by the faculty in question. Only that it is very unlikely. The fact that one can construct possible worlds where their behaviour is close to optimal is of rather little relevance to that.
Google did find that academic performance is no good predictor for job performance at Google.
Among people actually hired by Google. Who (1) pretty much all have very good academic performance (see e.g. this if it’s not clear why that’s relevant) and (2) will typically have been better in other respects if worse academically, in order to get hired: see e.g. this for more information.
I conjecture that, ironically, if Google measure again, they’ll find that GPA became a better predictor of job success when they stopped using it as an important metric for selecting candidates.
you argue as if scientific studies nearly always replicate
Not intentionally. I’m aware that they don’t. None the less, scientific studies are the best we have, and it’s not like there’s a shortage of studies finding evidence of the sort of sex bias we’re discussing.
None the less, scientific studies are the best we have
“Best we have” doesn’t justify a small confidence interval. If there no good evidence available on a topic the right thing to do is to be uncertain.
it’s not like there’s a shortage of studies finding evidence of the sort of sex bias we’re discussing
The default way to act in those situations is to form your opinions based on meta-analysis.
I conjecture that, ironically, if Google measure again, they’ll find that GPA became a better predictor of job success when they stopped using it as an important metric for selecting candidates.
You basically think that a bunch of highly paid staticians make a very trivial error when a lot of money is at stake. How confident are you in that prediction?
If there is no good evidence available on a topic the right thing to do is to be uncertain.
I agree. (Did I say something to suggest otherwise?)
The default way [...] is to form your opinions based on meta-analysis.
Given the time and inclination to do the meta-analysis (or someone else who’s already done the work), yes. Have you perchance done it or read the work of someone else who has?
I agree. (Did I say something to suggest otherwise?)
On this topic it seems like your position is that you know that employers act irrationally and don’t hire woman who would perform well.
My position is that I don’t know whether or not that’s a case. That means you have a smaller confidence interval. I consider the size of that interval unjustified.
Given the time and inclination to do the meta-analysis
In the absence of that work being done it’s not good to believe that one knows the answer.
My position is that I’ve seen an awful lot of evidence, both scientific and anecdotal, that seems best explained by supposing such irrationality. A few examples:
Another study of attitudes to hiring finding that for applicants early in their career just changing the name from female to male results in dramatically more positive assessment. (The differences were smaller with a candidate several years further into his/her career.)
A famous study by Goldberg submitted identical essays under male and female names and found that it got substantially better assessments with the male name. (I should add that this one seems to have been repeated several times, sometimes getting the same result and sometimes not. Different biases at different institutions?)
In each case, of course one can come up with explanations that don’t involve bias—as some commenters in this discussion have eagerly done. But it seems to me that the evidence is well past the point where denying the existence of sexist biases is one hell of a stretch.
By the same logic you could say that someone who hires people with high SRT scores engages in SRT bias. Someone who hires based on SRT scores could simply reasonably believe that people with high SRT scores are more competent.
Google’s HR department has a variety of factors on which it judges candidates. A few years afterwards they reevaluate their hiring decisions. They run a regression analysis and see which factors predict job performance at Google. They learn from that analysis and switch their hiring decision to hiring people which score highly on the factors that the regression analysis found predictive.
That’s how making rational hiring decisions looks like. In the process they found that college marks aren’t very relevant for predicting job performance. Being good at Fermi estimates unfortunately isn’t as well, so those LW people who train Fermi estimates don’t get benefits anymore when they want to get a job at Google.
Given current laws Google is not allowed to put values such as gender into the mix they use to make hiring decisions. That means that Google can’t make the hiring decisions that maximize predicted job performance.
The politics of the issue also make it pretty bad PR for them to publish results about the effects of a model that includes gender if the correct value in the regression analysis would mean worse chances for woman getting a job. It’s good PR for them if the correct value would mean to favor woman. No big company that does regression analysis on job performance published data that favoring in gender would mean hiring more woman. Factoring in gender into a regression analysis would mean that any bias against woman in subjective competence evaluations in interviews would be canceled by that factor.
Just imagine if a big company would find that by putting gender into their regression analysis they would hiring more women and get better average job performance as a result. Don’t you think those companies would lobby Washington to allow them to put gender into hiring decisions? The silence on the issue speaks.
It could be that the silencing of feminists who want to prevent “privileged” from talking about the issue is strong enough that rational companies don’t dare to speak about their need to change their hiring practices to hire more woman via making data driven arguments. If that’s the case that says a lot about the concept of privilege and it’s problem in shutting down rational arguments.
Imagine that academic performance has a really low value for predicting job performance. People that spend a lot of time preparing for tests get better academic marks. Woman spent more time than men preparing for academic tests. That means a woman of equal competence scores higher because she puts in more work. The test isn’t anymore a strict measure of competence but a measure of effort at scoring highly of the test. In that scenario it makes sense to infer that a woman with the same test score as a man is likely less competent as the man as long as you are hiring for “competence” and not for “putting in effort to game the test”.
If you write down the math you see that it depends on your priors for the effect size of how gender correlates with job performance.
Sure. It is possible to construct possible worlds in which the behaviour of the academic faculty investigated in this study is rational and unbiased and sensible and good. The question is: How credible is it that our world is one of them?
If you think it is at all credible, then I invite you to show me the numbers. Tell me what you think the actual relationship is between gender, academic performance, job performance, etc. Tell me why you think the numbers you’ve suggested are credible, and why they lead to the sort of results found in this study. Because my prediction is that to get the sort of results found in this study you will need to assume numbers that are really implausible. I could, of course, be wrong; in which case, show me. But I don’t think anything is achieved by reiterating that it’s possible for the results of this study to be consistent with good and unbiased (more precisely: “biased” only in the sense of recognizing genuine relevant correlations) decisions by the faculty. We all (I hope) know that already. “Possible” is waaaaay too low a bar.
Making wrong arguments isn’t good even if it leads to a true conclusion. I haven’t argued that the world happens to be shaped a certain way. I argue that your arguments are wrong. LessWrong is primarily a forum for rational debate. If you arguing for a position that I believe to be true but make arguments that are flawed I will object. That’s because arguments aren’t soldiers.
On the matter of the extend of gender discrimination I don’t have a fixed opinion. My uncertainty interval is pretty large. Not having a small uncertainty interval because you fall for flawed arguments matters. The fact that humans are by default overconfident is well replicated.
But if we become back to grades as a predictor: Google did find that academic performance is no good predictor for job performance at Google.
Of course Google won’t give you the relevant data as an academic does, but Google is a company that wants to make money. It actually has a stake in hiring high performing individuals.
While we are at it, you argue as if scientific studies nearly always replicate. We don’t live in a world where that’s true. Political debates tend to make people overconfident.
It looks to me as if that’s because you are treating them as if they are intended to be deductive inferences when in fact they are inductive ones.
At no point have I intended to argue that (e.g.) it is impossible that the results found in this study are the result of accurate rational evaluation by the faculty in question. Only that it is very unlikely. The fact that one can construct possible worlds where their behaviour is close to optimal is of rather little relevance to that.
Among people actually hired by Google. Who (1) pretty much all have very good academic performance (see e.g. this if it’s not clear why that’s relevant) and (2) will typically have been better in other respects if worse academically, in order to get hired: see e.g. this for more information.
I conjecture that, ironically, if Google measure again, they’ll find that GPA became a better predictor of job success when they stopped using it as an important metric for selecting candidates.
Not intentionally. I’m aware that they don’t. None the less, scientific studies are the best we have, and it’s not like there’s a shortage of studies finding evidence of the sort of sex bias we’re discussing.
“Best we have” doesn’t justify a small confidence interval. If there no good evidence available on a topic the right thing to do is to be uncertain.
The default way to act in those situations is to form your opinions based on meta-analysis.
You basically think that a bunch of highly paid staticians make a very trivial error when a lot of money is at stake. How confident are you in that prediction?
I agree. (Did I say something to suggest otherwise?)
Given the time and inclination to do the meta-analysis (or someone else who’s already done the work), yes. Have you perchance done it or read the work of someone else who has?
Not very.
[EDITED to fix a punctuation typo]
On this topic it seems like your position is that you know that employers act irrationally and don’t hire woman who would perform well. My position is that I don’t know whether or not that’s a case. That means you have a smaller confidence interval. I consider the size of that interval unjustified.
In the absence of that work being done it’s not good to believe that one knows the answer.
My position is that I’ve seen an awful lot of evidence, both scientific and anecdotal, that seems best explained by supposing such irrationality. A few examples:
The study we’ve been discussing here.
A neurobiologist transitions from female to male and is immediately treated as much more competent.
Another study of attitudes to hiring finding that for applicants early in their career just changing the name from female to male results in dramatically more positive assessment. (The differences were smaller with a candidate several years further into his/her career.)
A famous study by Goldberg submitted identical essays under male and female names and found that it got substantially better assessments with the male name. (I should add that this one seems to have been repeated several times, sometimes getting the same result and sometimes not. Different biases at different institutions?)
Auditioning orchestral players behind a screen makes women do much better relative to men.
In each case, of course one can come up with explanations that don’t involve bias—as some commenters in this discussion have eagerly done. But it seems to me that the evidence is well past the point where denying the existence of sexist biases is one hell of a stretch.