Thanks, Christian! I’ve actually been in communication with Spencer and Stefan—they’ve been an immensely helpful resource. We have both stumbled upon a similar, difficult problem. Namely, it’s very rare to be “certain” about the controversial questions (which is the point of the contest). By coincidence, a lot of the questions used in their political bias tests I also came up with for my intervention, but some of the others are actually less straightforwardly verifiable than is let on by the test.
It seems hard to be certain what the true probability happens to be that global warming increases the frequency of floods. On the other hand it’s easy to be certain about the probability that the latest IPCC report gives to that event.
In a similar sense you can ask about “How many percent of Muslims in the UK believe according to the GSS that homosexuality should be punishable by death?”
When talking about deaths due to Chernobyl you can argue whether you should or shouldn’t count those who got depressive and committed suicide afterwards but you get a clear number from the UNSCEAR assessments of the Chernobyl accident.
If you word your questions to say “What does authoritative source X say about Y”, you can have certain answers.
There’s prior art.
Stefan Schubert created along with Spencer Greenberg created along with https://programs.clearerthinking.org/political_bias_test.html (https://www.lesswrong.com/posts/3jeRSuLopjjFwFpoG/political-debiasing-and-the-political-bias-test. There was a LessWrong discussion beforehand that informed the questions on that test.
https://www.lesswrong.com/posts/3jeRSuLopjjFwFpoG/political-debiasing-and-the-political-bias-test is an article about his test.
He wrote a guide for creating new tests:
https://docs.google.com/document/d/1OIiIVrNwK6ftRZqLfEmhwXHj0tx-Uhg_N2kfBEFpuP0/edit
Thanks, Christian! I’ve actually been in communication with Spencer and Stefan—they’ve been an immensely helpful resource. We have both stumbled upon a similar, difficult problem. Namely, it’s very rare to be “certain” about the controversial questions (which is the point of the contest). By coincidence, a lot of the questions used in their political bias tests I also came up with for my intervention, but some of the others are actually less straightforwardly verifiable than is let on by the test.
It seems hard to be certain what the true probability happens to be that global warming increases the frequency of floods. On the other hand it’s easy to be certain about the probability that the latest IPCC report gives to that event.
In a similar sense you can ask about “How many percent of Muslims in the UK believe according to the GSS that homosexuality should be punishable by death?”
When talking about deaths due to Chernobyl you can argue whether you should or shouldn’t count those who got depressive and committed suicide afterwards but you get a clear number from the UNSCEAR assessments of the Chernobyl accident.
If you word your questions to say “What does authoritative source X say about Y”, you can have certain answers.