3) Which of the following statements best describes expert scientists’ views of the claim that humans are causing mass extinction of species at a rate that is at least 100 times the natural rate? (Same possible answers)
Without knowing anything about the extinction of species, I could guess that the answer is “most scientists agree”.
If the correct number is not 100 but is large, the question would incorrectly conclude that some people who are not biased are biased (since someone who falsely thinks the number is 100 when it’s really 75 or 125 is not biased, but would incorrectly answer “scientists agree”).
If the correct number is not 100 but is small, the question would incorrectly conclude that some people who are biased are not biased (since someone who falsely thinks the number is 75 or 125 when it’s really 1 is biased, but this bias would be undetectable since he would correctly answer the question with “scientists disagree”)
Therefore the correct number is 100.
This question should be phrased using words like “many”, not using the exact number 100.
Generally, however, people should be suspicious of such questions because in real political discourse, these questions are used to set the terms of the debate.
-- Does it matter whether the death penalty increases homicide rates?
-- Does it matter that humans cause climate change regardless of the size of the change?
-- I would think that any truly expert economist would say “we have no way to know the answers to these questions to the same degree as we know physics or chemistry answers. I could give you my educated opinion, but there’s still a lot of disagreement within economics”.
Hm, you’re right that question 3) is not formulated rightly. Great comment!
Thanks for the poll data. One worry is, though, that the police officers might not be seen as proper “experts” in the same sense as climate scientists are. I need to think about that.
The data are very interesting, though. US police officers seem to very conservative indeed.
Imagine that you don’t know much about homeopathy, but you do know that experts oppose it. Then someone asks you the question “The number of homeopathic cures of all types rejected by the FDA for not being effective is (much less than) (less than) (equal to) (greater than) (much greater than) the number of allopathic cancer cures.”
If you approached this question out of context, you would think “I know that experts believe homeopathy isn’t effective. The FDA uses experts. So experts probably rejected lots of homeopathic remedies.”
If you approached this question in context, however, you would reason “I know that experts believe homeopathy isn’t effective. But given the way this question is phrased, it’s being asked by a homeopath. He’s probably asking this question because it makes homeopathy look good, so this must be an unusual situation where experts’ belief on homeopathy doesn’t affect the answer, and he’s falsely trying to imply that it does. So the FDA probably rejected few homeopathic remedies for being ineffective, but for some reason this doesn’t reflect the belief of experts.”
Actually, one of the sponsors of the act that created the FDA was a homeopath and he wrote in an exception for homeopathy, so homeopathic treatments don’t have to prove they are safe and effective.
Also, keep note of who this question would falsely mark as biased. Someone who opposes homeopathy and correctly knows that experts also oppose homeopathy, who tries to reason the first way, would be marked down as biased, because he answered in a way favorable to his own position but contrary to the facts. Yet answering the first way doesn’t mean bias, it just means he ignored the agenda of the person asking the question.
Without knowing anything about the extinction of species, I could guess that the answer is “most scientists agree”.
If the correct number is not 100 but is large, the question would incorrectly conclude that some people who are not biased are biased (since someone who falsely thinks the number is 100 when it’s really 75 or 125 is not biased, but would incorrectly answer “scientists agree”).
If the correct number is not 100 but is small, the question would incorrectly conclude that some people who are biased are not biased (since someone who falsely thinks the number is 75 or 125 when it’s really 1 is biased, but this bias would be undetectable since he would correctly answer the question with “scientists disagree”)
Therefore the correct number is 100.
This question should be phrased using words like “many”, not using the exact number 100.
For pro-conservative questions, one could be: In a recent poll of 15000 police officers polled, a large majority thought assault weapon bans are effective, a small majority thought assault weapon bans are effective, they were about equal, a small majority thought assault weapon bans are ineffective, a large majority thought assault weapon bans are ineffective. http://www.policeone.com/Gun-Legislation-Law-Enforcement/articles/6183787-PoliceOnes-Gun-Control-Survey-11-key-lessons-from-officers-perspectives/
Generally, however, people should be suspicious of such questions because in real political discourse, these questions are used to set the terms of the debate.
-- Does it matter whether the death penalty increases homicide rates?
-- Does it matter that humans cause climate change regardless of the size of the change?
-- I would think that any truly expert economist would say “we have no way to know the answers to these questions to the same degree as we know physics or chemistry answers. I could give you my educated opinion, but there’s still a lot of disagreement within economics”.
Hm, you’re right that question 3) is not formulated rightly. Great comment!
Thanks for the poll data. One worry is, though, that the police officers might not be seen as proper “experts” in the same sense as climate scientists are. I need to think about that.
The data are very interesting, though. US police officers seem to very conservative indeed.
Thanks!
The point of the exercise is detecting bias. As such it’s not important whether the answer to the question is important.
It doesn’t work that way.
Imagine that you don’t know much about homeopathy, but you do know that experts oppose it. Then someone asks you the question “The number of homeopathic cures of all types rejected by the FDA for not being effective is (much less than) (less than) (equal to) (greater than) (much greater than) the number of allopathic cancer cures.”
If you approached this question out of context, you would think “I know that experts believe homeopathy isn’t effective. The FDA uses experts. So experts probably rejected lots of homeopathic remedies.”
If you approached this question in context, however, you would reason “I know that experts believe homeopathy isn’t effective. But given the way this question is phrased, it’s being asked by a homeopath. He’s probably asking this question because it makes homeopathy look good, so this must be an unusual situation where experts’ belief on homeopathy doesn’t affect the answer, and he’s falsely trying to imply that it does. So the FDA probably rejected few homeopathic remedies for being ineffective, but for some reason this doesn’t reflect the belief of experts.”
For instance, if most homeopathic treatments are not submitted to the FDA, they would not have a chance to reject them.
Actually, one of the sponsors of the act that created the FDA was a homeopath and he wrote in an exception for homeopathy, so homeopathic treatments don’t have to prove they are safe and effective.
Also, keep note of who this question would falsely mark as biased. Someone who opposes homeopathy and correctly knows that experts also oppose homeopathy, who tries to reason the first way, would be marked down as biased, because he answered in a way favorable to his own position but contrary to the facts. Yet answering the first way doesn’t mean bias, it just means he ignored the agenda of the person asking the question.