Analogy: I describe a real-life situation of a police officer shooting a suspect. I then ask people what they think the race of the police officer and suspect are. Because I am referring to a specific real-life case, the question has a single, factual, answer, and people’s answer is either correct or not.
Yet I can manipulate the question to show liberal bias or conservative bias, my choice, simply by which case I choose to ask the question about.
The best way to ask that question to legitimately detect bias would be to choose a typical case, and to assume that people who haven’t heard of the specific case would answer depending on the facts about typical cases.
And in this situation, a typical case would be an index of X that accurately measures X. Choosing an index that doesn’t accurately measure X would skew the ability to use that question to detect bias, since I expect that unbiased people who haven’t heard of the index in question would answer based on an accurate measure of X.
That’s an interesting one—I think black people are disproportionately at risk of being killed by the police in the US, but about as many white people as black get killed.
Analogy: I describe a real-life situation of a police officer shooting a suspect. I then ask people what they think the race of the police officer and suspect are. Because I am referring to a specific real-life case, the question has a single, factual, answer, and people’s answer is either correct or not.
Yet I can manipulate the question to show liberal bias or conservative bias, my choice, simply by which case I choose to ask the question about.
The best way to ask that question to legitimately detect bias would be to choose a typical case, and to assume that people who haven’t heard of the specific case would answer depending on the facts about typical cases.
And in this situation, a typical case would be an index of X that accurately measures X. Choosing an index that doesn’t accurately measure X would skew the ability to use that question to detect bias, since I expect that unbiased people who haven’t heard of the index in question would answer based on an accurate measure of X.
That’s an interesting one—I think black people are disproportionately at risk of being killed by the police in the US, but about as many white people as black get killed.