Inspired by your final paragraph, I sought out a variety of test questions on the web—both on Steven’s blog and elsewhere. I was expecting systematic overconfidence, with a smaller chance of systematic underconfidence, throughout the probability spectrum.
Instead I found a very interesting pattern.
When I was 90% or 95% certain of a fact, I was slightly overconfident. My 90% estimates shook out at about 80%, and my 95% estimates shook out around 90%. When I was completely uncertain of a fact, I was also slightly overconfident, but within the realm of experimental error.
But when I was just 50% confident of a fact, I was almost always wrong. Far more often than anyone could achieve by random guessing: my wrongness was thorough and integrated and systematic.
Clearly, that feeling of slight concern which I’ve always interpreted as, “I think I remember X, but it could go either way,” actually means something closer to, “X is not true; my beliefs are inconsistent.”
If I’m sure I know something, I probably do. If I’m sure I’m clueless, I probably am. But if I think I might know something, then I almost certainly have it backwards.
Is this a common bias which I should have read about by now?
Inspired by your final paragraph, I sought out a variety of test questions on the web—both on Steven’s blog and elsewhere. I was expecting systematic overconfidence, with a smaller chance of systematic underconfidence, throughout the probability spectrum.
Instead I found a very interesting pattern.
When I was 90% or 95% certain of a fact, I was slightly overconfident. My 90% estimates shook out at about 80%, and my 95% estimates shook out around 90%. When I was completely uncertain of a fact, I was also slightly overconfident, but within the realm of experimental error.
But when I was just 50% confident of a fact, I was almost always wrong. Far more often than anyone could achieve by random guessing: my wrongness was thorough and integrated and systematic.
Clearly, that feeling of slight concern which I’ve always interpreted as, “I think I remember X, but it could go either way,” actually means something closer to, “X is not true; my beliefs are inconsistent.”
If I’m sure I know something, I probably do. If I’m sure I’m clueless, I probably am. But if I think I might know something, then I almost certainly have it backwards.
Is this a common bias which I should have read about by now?
Interesting!
By the way, HTML tags don’t work here; click “Help” to the lower right of the edit window to see the Markup syntax rules.
Thanks—edited for proper italics.