I notice you have changed the article to use the name “positive bias”, but you are still describing it as if it were the confirmation bias, “people look for confirming rather than disconfirming evidence”. That is not what is going on. They don’t avoid testing the sequence 10,13,14 because it would disconfirm their theory, they don’t even think of testing it because their theory doesn’t say it is a magic sequence and they want to test magic sequences. They don’t even notice that their theory of what is a magic sequence is also implicitly a theory of what is not a magic sequence, and don’t realize they should also test that the theoretically non magic sequences are in fact not magic.
This seems like the clearest way to describe the case in English. The problem in the 2, 4, 6 problem is exactly that people test sequences that would confirm rather than disconfirm their theory. The problem is exactly that they look for confirming rather than disconfirming evidence.
The distinction Eliezer makes is between the tendency to seek what he calls positive and negative cases (positive bias) and the tendency to seek to preserve our original beliefs (confirmation bias).
By this light, I’m talking about positive bias. I don’t at any point say that the participant is seeking confirming rather than disconfirming evidence in order to preserve their beliefs. Instead, I’m simply noting a bias toward searching for a certain sort of test case.
But I think that those test cases are more clearly described as confirming or disconfirming cases rather than positive or negative cases. Everyone immediately knows what a confirming case is. On the other hand, I don’t think the word positive is immediately clear.
So I changed the name from confirmation bias to positive bias to individuate which of the two biases I’m talking about but I still think the bias is best explained in the language of confirmation and disconfirmation.
So I agree that they don’t avoid testing the sequence 10, 13, 14 because it would disconfirm their theory (and they want to maintain their original belief) but I do think that people don’t think of testing disconfirming sequences because such sequences are not magic sequences suggested by their theory.
It seems that we disagree on which language is clearest. My opinion: It’s only the prior debate over this topic on Less Wrong that makes the word confirming seem confusing. Most newcomers to the idea would find the word confirming more clear here. Old timers will understand these issues anyway so I want to explain clearly to newcomers.
Ideally, I’d prefer everyone to understand it, of course. Rather than keeping editing it, I’ll post my latest paragraph here and, if you have time, I’d be interested to know whether you think this works better. If so, I’ll edit when I’m back at the computer (heading out soon)
I then used the exercise to explain a bias called positive bias. First, I noted that only 21% of respondents reached the right answer to this scenario. Then I pointed out that the interesting point isn’t this figure but rather why so few people reach the right answer. Specifically, people think to test positive, rather than negative, cases. In other words, they’re more likely to test cases that their theory predicts will occur (in this case, those that get a yes answer) then cases that their theory predicts won’t. So if someone’s initial theory was that the rule was, “three numbers, each two higher than the previous one” then they might test “10, 12, 14“ as this is a positive case for their theory. On the other hand, they probably wouldn’t test “10, 14, 12” or “10, 13, 14” as these are negative cases for their prediction of the rule.
This demonstrates positive bias—the bias toward thinking to test positive, rather then negative, cases for their theory.
I notice you have changed the article to use the name “positive bias”, but you are still describing it as if it were the confirmation bias, “people look for confirming rather than disconfirming evidence”. That is not what is going on. They don’t avoid testing the sequence 10,13,14 because it would disconfirm their theory, they don’t even think of testing it because their theory doesn’t say it is a magic sequence and they want to test magic sequences. They don’t even notice that their theory of what is a magic sequence is also implicitly a theory of what is not a magic sequence, and don’t realize they should also test that the theoretically non magic sequences are in fact not magic.
This seems like the clearest way to describe the case in English. The problem in the 2, 4, 6 problem is exactly that people test sequences that would confirm rather than disconfirm their theory. The problem is exactly that they look for confirming rather than disconfirming evidence.
The distinction Eliezer makes is between the tendency to seek what he calls positive and negative cases (positive bias) and the tendency to seek to preserve our original beliefs (confirmation bias).
By this light, I’m talking about positive bias. I don’t at any point say that the participant is seeking confirming rather than disconfirming evidence in order to preserve their beliefs. Instead, I’m simply noting a bias toward searching for a certain sort of test case.
But I think that those test cases are more clearly described as confirming or disconfirming cases rather than positive or negative cases. Everyone immediately knows what a confirming case is. On the other hand, I don’t think the word positive is immediately clear.
So I changed the name from confirmation bias to positive bias to individuate which of the two biases I’m talking about but I still think the bias is best explained in the language of confirmation and disconfirmation.
So I agree that they don’t avoid testing the sequence 10, 13, 14 because it would disconfirm their theory (and they want to maintain their original belief) but I do think that people don’t think of testing disconfirming sequences because such sequences are not magic sequences suggested by their theory.
It seems that we disagree on which language is clearest. My opinion: It’s only the prior debate over this topic on Less Wrong that makes the word confirming seem confusing. Most newcomers to the idea would find the word confirming more clear here. Old timers will understand these issues anyway so I want to explain clearly to newcomers.
Ideally, I’d prefer everyone to understand it, of course. Rather than keeping editing it, I’ll post my latest paragraph here and, if you have time, I’d be interested to know whether you think this works better. If so, I’ll edit when I’m back at the computer (heading out soon)