As Douglas_Knight points out, it’s only 10⁄12, a probability of ~0.016. In a sample of ~50 we should see about one person at that level of accuracy or inaccuracy, which is exactly what we see. I’m no more inclined to give #14 a medal than I am to call #43 a dunce. See the histogram I stuck on to the end of the post for more intuition about why I see these extreme results as normal.
I absolutely will fess up to exaggerating in that sentence for the sake of dramatic effect. Some judges, such as yourself, were MUCH less wrong. I hope you don’t mind me outing you as one of the people who got a positive score, and that’s a reflection of you being better calibrated. That said, if you say “I’m 70% confident” four times, and only get it right twice, that’s evidence that you were still (slightly) overconfident when you thought “decently able to discern genuine writing from fakery”.
I’m #43 and I’ll accept my dunce cap. I responded just after I began lurking here. I remember having little confidence in my responses and yet I apparently answered as if I did. I really have no insight into why I answered this way. My cringeworthy results reinforce to me the importance of sticking around and improving my thinking.
that’s a reflection of you being better calibrated
Or, of course, just lucky. If you aren’t giving #14 a medal, you shouldn’t be giving me one either. (Though, as it happens, I have some reason to think my calibration is pretty good.) And yes, I was still slightly overconfident, and my intention in what I wrote above was to make it clear that I recognize that.
As Douglas_Knight points out, it’s only 10⁄12, a probability of ~0.016. In a sample of ~50 we should see about one person at that level of accuracy or inaccuracy, which is exactly what we see. I’m no more inclined to give #14 a medal than I am to call #43 a dunce. See the histogram I stuck on to the end of the post for more intuition about why I see these extreme results as normal.
I absolutely will fess up to exaggerating in that sentence for the sake of dramatic effect. Some judges, such as yourself, were MUCH less wrong. I hope you don’t mind me outing you as one of the people who got a positive score, and that’s a reflection of you being better calibrated. That said, if you say “I’m 70% confident” four times, and only get it right twice, that’s evidence that you were still (slightly) overconfident when you thought “decently able to discern genuine writing from fakery”.
I’m #43 and I’ll accept my dunce cap. I responded just after I began lurking here. I remember having little confidence in my responses and yet I apparently answered as if I did. I really have no insight into why I answered this way. My cringeworthy results reinforce to me the importance of sticking around and improving my thinking.
Or, of course, just lucky. If you aren’t giving #14 a medal, you shouldn’t be giving me one either. (Though, as it happens, I have some reason to think my calibration is pretty good.) And yes, I was still slightly overconfident, and my intention in what I wrote above was to make it clear that I recognize that.