Eliezer described “positive bias” (which I’ll rename “positive test bias” for reasons explained in Unnamed’s comment below) in an LW post and an HPMOR chapter. According to him, dealing with that bias requires a kind of mental gymnastics that doesn’t come naturally to most people: “twisty negative thinking” or “flinching toward zero”. You’re supposed to devise tests that come out false if your hypothesis is true. It’s a bit confusing.
I think there’s a simpler way to think about it. Positive test bias is just our bias toward strong hypotheses. You can deal with it by asking yourself, how can I test if my hypothesis is too strong?
The LW post about the bias mentions the Wason 2-4-6 task. If you’re told that the number sequence 2-4-6 has property X, it’s tempting to guess that property X means “ascending arithmetic progression”. To test if that hypothesis is too strong, you need to try some other sequences that are arithmetic progressions but not ascending, and some that are ascending but not arithmetic progressions. Easy!
The HPMOR chapter describes Hermione spilling some soda on her robe, and a moment later it mysteriously becomes clean again. Hermione comes up with a hypothesis that her robe is magically self-cleaning. To test if that hypothesis is too strong, she can try spilling something else on her robe. There’s no need for any counterintuitive thinking.
That technique is useful in other areas as well. For example, I often get carried away when writing posts, and end up with a draft full of wide-reaching conclusions. But then I ask myself, doesn’t that sound a bit too strong? So I look for counterexamples, and the point I’m trying to make either dissolves or becomes much more robust.
Our bias toward strong hypotheses is especially harmful in politics. We will defend a hypothesis like “group X is to blame for everything” for its explanatory power, never noticing the real problem—that it’s too strong. We’d all benefit from recognizing when it happens, and weakening our hypotheses until they match reality.
A simpler way to think about positive test bias
Eliezer described “positive bias” (which I’ll rename “positive test bias” for reasons explained in Unnamed’s comment below) in an LW post and an HPMOR chapter. According to him, dealing with that bias requires a kind of mental gymnastics that doesn’t come naturally to most people: “twisty negative thinking” or “flinching toward zero”. You’re supposed to devise tests that come out false if your hypothesis is true. It’s a bit confusing.
I think there’s a simpler way to think about it. Positive test bias is just our bias toward strong hypotheses. You can deal with it by asking yourself, how can I test if my hypothesis is too strong?
The LW post about the bias mentions the Wason 2-4-6 task. If you’re told that the number sequence 2-4-6 has property X, it’s tempting to guess that property X means “ascending arithmetic progression”. To test if that hypothesis is too strong, you need to try some other sequences that are arithmetic progressions but not ascending, and some that are ascending but not arithmetic progressions. Easy!
The HPMOR chapter describes Hermione spilling some soda on her robe, and a moment later it mysteriously becomes clean again. Hermione comes up with a hypothesis that her robe is magically self-cleaning. To test if that hypothesis is too strong, she can try spilling something else on her robe. There’s no need for any counterintuitive thinking.
That technique is useful in other areas as well. For example, I often get carried away when writing posts, and end up with a draft full of wide-reaching conclusions. But then I ask myself, doesn’t that sound a bit too strong? So I look for counterexamples, and the point I’m trying to make either dissolves or becomes much more robust.
Our bias toward strong hypotheses is especially harmful in politics. We will defend a hypothesis like “group X is to blame for everything” for its explanatory power, never noticing the real problem—that it’s too strong. We’d all benefit from recognizing when it happens, and weakening our hypotheses until they match reality.