A link or button to flip your last right/wrong would be nice. I had assigned 0% confidence for one question and accidentally said I got it right. Misclicks aren’t the same as poor calibration.
Also, a little more on how to use it would make sense—the first one or two I did, I thought it was, ‘how confident are you that this assertion is true’ and I thought it was very oddly phrased. Then I realized.
What would help most is:
“Pick an answer. How confident are you that your answer is correct?”
Then, make sure that when the user clicks the ‘show answer’ button, make sure that neither of the two new buttons are in the same place.
ALSO, it would be nice if the calibration curve showed the credible interval for each bin, so I can tell at a glance that my getting 1⁄1 right at 30% and 0⁄1 right at 60% isn’t actually that big a hit to my calibration.
And if the second graph was stacked so that I don’t have this giant red bar at 100%, which just looks odd. If it was red behind/on-top-of green, that would make the most sense (if stacked on top, you will obviously need to take the difference to maintain the sense of the graph).
Do you intend to curate out questions that are impossible/require additional clarifications like Alex would have given in advance or people would have worked out from the easy ones?
A link or button to flip your last right/wrong would be nice. I had assigned 0% confidence for one question and accidentally said I got it right. Misclicks aren’t the same as poor calibration.
Also, a little more on how to use it would make sense—the first one or two I did, I thought it was, ‘how confident are you that this assertion is true’ and I thought it was very oddly phrased. Then I realized.
Got it. I’ll make them color coded and farther apart.
I’ll write some better instructions as well.
What would help most is: “Pick an answer. How confident are you that your answer is correct?”
Then, make sure that when the user clicks the ‘show answer’ button, make sure that neither of the two new buttons are in the same place.
ALSO, it would be nice if the calibration curve showed the credible interval for each bin, so I can tell at a glance that my getting 1⁄1 right at 30% and 0⁄1 right at 60% isn’t actually that big a hit to my calibration.
And if the second graph was stacked so that I don’t have this giant red bar at 100%, which just looks odd. If it was red behind/on-top-of green, that would make the most sense (if stacked on top, you will obviously need to take the difference to maintain the sense of the graph).
Do you intend to curate out questions that are impossible/require additional clarifications like Alex would have given in advance or people would have worked out from the easy ones?