I don’t see how that is confirmation bias. Where does the beginning programmer discount or ignore disconfirming evidence? If the print statement shows the troublesome variable has the correct value following the suspect line, that is evidence against the suspect line being the bug.
The problem in this case is that programmer is paying attention to only part of the meaning of that line being a bug. If it is a bug, it would transform correct state to incorrect state, and the programmer is only validating the resulting incorrect state, not the preceding correct state.
Though, I will sometimes add a single debug statement, not to test some particular line of code, but to test if the bug is before or after the debug statement, so I can look more closely at the offending section and narrow it down further.
Yep. Actually on meta level I see more confirmation bias right here. For instance gwern doesn’t seek to disprove that it is confirmation bias, but seeks to confirm it, and sees how the print statement after suspect line wouldn’t disconfirm the bug being on that line if the bug is above that line, but doesn’t see that it would disconfirm the bug if the bug was below suspect line. (and in any case if you want to conclusively show that there is a bug on suspect line, you need to understand what the bug is, precisely, which is confirming evidence, and which requires you to know the state after and see if it matches what you think the bug would produce.)
I do imagine that confirmation bias does exist in programming whenever the programmer did not figure out or learn how to do it right, but learning that there’s this bias, and learning how to do it right, are different things entirely. The programmer that learned how to debug can keep track of set of places where the bug could be, update it correctly, and seek effective ways to narrow that down.
Maybe it isn’t confirmation bias. But
Wikipedia says that
“[p]eople display [confirmation] bias when they gather or remember
information selectively, or when they interpret it in a biased way”. If that’s
a good description, then gwern’s example would fall under “gather[ing]”.
The bias the programmer in gwern’s example is exhibiting is the same one that
makes people fail the Wason selection
task—searching for
confirming evidence rather than disconfirming evidence.
Edit: Retracting this because I think
JGWeissman is right.
If that’s a good description, then gwern’s example would fall under “gather[ing]”.
The programmer is indeed gathering evidence, but I don’t see how they are gathering evidence in a way that is meant to produce confirming rather than disconfirming evidence. As I have explained, the test could show a correct value for the troublesome variable and be evidence against the suspect line being the bug. The test will be somewhat biased towards confirmation in that it is really testing if the suspect line has a bug or any earlier line has a bug, but I don’t think this bias reflects the programmer seeking only confirming evidence so much as not understanding what they are testing.
The bias the programmer is exhibiting in gwern’s example is the same one that makes people fail the Wason selection task—searching for confirming evidence rather than disconfirming evidence.
That is not the cause of failure in the Wason selection task. The problem is not using contrapositives, that is, realizing “even implies red” is logically equivalent to its contrapositive “not red implies not even”, so to test “even implies red” you have to check cards that show an even number or a non red color.
This is similar to the failure to use contrapositives behind the Positive Test Bias, which is itself similar to gwern’s example in that it involves failure to test every aspect of the hypothesis.
I don’t see how that is confirmation bias. Where does the beginning programmer discount or ignore disconfirming evidence? If the print statement shows the troublesome variable has the correct value following the suspect line, that is evidence against the suspect line being the bug.
The problem in this case is that programmer is paying attention to only part of the meaning of that line being a bug. If it is a bug, it would transform correct state to incorrect state, and the programmer is only validating the resulting incorrect state, not the preceding correct state.
Though, I will sometimes add a single debug statement, not to test some particular line of code, but to test if the bug is before or after the debug statement, so I can look more closely at the offending section and narrow it down further.
Yep. Actually on meta level I see more confirmation bias right here. For instance gwern doesn’t seek to disprove that it is confirmation bias, but seeks to confirm it, and sees how the print statement after suspect line wouldn’t disconfirm the bug being on that line if the bug is above that line, but doesn’t see that it would disconfirm the bug if the bug was below suspect line. (and in any case if you want to conclusively show that there is a bug on suspect line, you need to understand what the bug is, precisely, which is confirming evidence, and which requires you to know the state after and see if it matches what you think the bug would produce.)
I do imagine that confirmation bias does exist in programming whenever the programmer did not figure out or learn how to do it right, but learning that there’s this bias, and learning how to do it right, are different things entirely. The programmer that learned how to debug can keep track of set of places where the bug could be, update it correctly, and seek effective ways to narrow that down.
Maybe it isn’t confirmation bias. But Wikipedia says that “[p]eople display [confirmation] bias when they gather or remember information selectively, or when they interpret it in a biased way”. If that’s a good description, then gwern’s example would fall under “gather[ing]”.
The bias the programmer in gwern’s example is exhibiting is the same one that makes people fail the Wason selection task—searching for confirming evidence rather than disconfirming evidence.
Edit: Retracting this because I think JGWeissman is right.
The programmer is indeed gathering evidence, but I don’t see how they are gathering evidence in a way that is meant to produce confirming rather than disconfirming evidence. As I have explained, the test could show a correct value for the troublesome variable and be evidence against the suspect line being the bug. The test will be somewhat biased towards confirmation in that it is really testing if the suspect line has a bug or any earlier line has a bug, but I don’t think this bias reflects the programmer seeking only confirming evidence so much as not understanding what they are testing.
That is not the cause of failure in the Wason selection task. The problem is not using contrapositives, that is, realizing “even implies red” is logically equivalent to its contrapositive “not red implies not even”, so to test “even implies red” you have to check cards that show an even number or a non red color.
This is similar to the failure to use contrapositives behind the Positive Test Bias, which is itself similar to gwern’s example in that it involves failure to test every aspect of the hypothesis.
You’re right. I’m retracting the grandparent.