Boy, lots of the remote learning stuff does suffer from predicting 2019 instead of 2020...
I suggest running inter-rater sanity checks and sometimes asking the rater if their answers should be inverted; I bet that at least one reader will misremember the instructions as “how true on a scale of 1 to 5” instead of your intended “how false on a scale of 1 to 5″.
lots of the remote learning stuff does suffer from predicting 2019 instead of 2020
I wouldn’t call it a successful prediction anyway—he predicts this to be the normal state of affairs, whereas the current situation is a temporary reaction to extraordinary circumstances
Boy, lots of the remote learning stuff does suffer from predicting 2019 instead of 2020...
I suggest running inter-rater sanity checks and sometimes asking the rater if their answers should be inverted; I bet that at least one reader will misremember the instructions as “how true on a scale of 1 to 5” instead of your intended “how false on a scale of 1 to 5″.
I wouldn’t call it a successful prediction anyway—he predicts this to be the normal state of affairs, whereas the current situation is a temporary reaction to extraordinary circumstances
I know. I hesitated, but decided to go for 2019 because we’re unsure what will persist beyond 2020, of the current changes.
Yep. I’ll find questions with near consensus, and check whether some people are systematically inverting those.