Yayy! I was having a shitty day, and seeing these results posted lifted my spirits. Thank you for that! Below are my assorted thoughts:
I’m a little disappointed that the correlation between height and P(supernatural)-and-similar didn’t hold up this year, because it was really fun trying to come up with explanations for that that weren’t prima facie moronic. Maybe that should have been a sign it wasn’t a real thing.
The digit ratio thing is indeed delicious. I love that stuff. I’m surprised there wasn’t a correlation to sexual orientation, though, since I seem to recall reading that that was relatively well-supported. Oh well.
WTF was going on with the computer games question? Could there have been some kind of widespread misunderstanding of the question? In any case, it’s pretty clearly poorly-calibrated Georg, but the results from the other questions are horrendous enough on their own.
On that subject, I have to say that even more amusing than the people who gave 100% and got it wrong are the people who put down 0% and then got it right—aka, really lucky guessers :P
I remember answering the computer games question and at first feeling like I knew the answer. Then I realized the feeling I was having was that I had a better shot at the question than the average person that I knew, not that I knew the answer with high confidence. Once I mentally counted up all the games that I thought might be it, then considered all the games I probably hadn’t even thought of (of which Minecraft was one), I realized I had no idea what the right answer was and put something like 5% confidence in The Sims 3 (which at least is a top ten game). But the point is that I think I almost didn’t catch my mistake before it was too late, and this kind of error may be common.
I was confident in my incorrect computer game answer because I had recently read this Wikipedia page List of best-selling video games remembered the answer and unthinkingly assumed that “video games” was the same as “computer games”.
The correct answer is Tetris. The question should have been what is the best selling personal computer game of all time? Mobile phones are technically computers too. I’m not sure how much difference that would have made.
I interpreted the question to include mobile devices and answered Tetris with high confidence.
It would be interesting to see the results of the question if we accepted either Tetris or Minecraft as the correct answer, since both are correct depending on whether or not “computer” was meant to mean “IBM PC Compatible” or “video game playing platform”
On the computer game question: Isn’t there an implicit “X is true and X will be marked correct by the rater”? You’d hope these two are clearly aligned, but if you’ve taken many real-world quizzes, you’ll recognize the difference.
I think the computer games question has to do with tribal identity-people who love a particularly well known game might be more inclined to list it as being the best seller ever and put down higher confidence because they love it so much.
Kind of like owners of Playstations and Xboxs will debate the superiority of their technical specs regardless of whether they’re superior or not.
I think the computer games result has to do with it being a bad question. There are many legitimate answers depending on how you interpret the question, including my answer that Minesweeper sells as a bundle with Windows and thus has probably sold more copies than anything else.
Yayy! I was having a shitty day, and seeing these results posted lifted my spirits. Thank you for that! Below are my assorted thoughts:
I’m a little disappointed that the correlation between height and P(supernatural)-and-similar didn’t hold up this year, because it was really fun trying to come up with explanations for that that weren’t prima facie moronic. Maybe that should have been a sign it wasn’t a real thing.
The digit ratio thing is indeed delicious. I love that stuff. I’m surprised there wasn’t a correlation to sexual orientation, though, since I seem to recall reading that that was relatively well-supported. Oh well.
WTF was going on with the computer games question? Could there have been some kind of widespread misunderstanding of the question? In any case, it’s pretty clearly poorly-calibrated Georg, but the results from the other questions are horrendous enough on their own.
On that subject, I have to say that even more amusing than the people who gave 100% and got it wrong are the people who put down 0% and then got it right—aka, really lucky guessers :P
Congrats to the Snicket fan!
This was a good survey and a good year. Cheers!
I remember answering the computer games question and at first feeling like I knew the answer. Then I realized the feeling I was having was that I had a better shot at the question than the average person that I knew, not that I knew the answer with high confidence. Once I mentally counted up all the games that I thought might be it, then considered all the games I probably hadn’t even thought of (of which Minecraft was one), I realized I had no idea what the right answer was and put something like 5% confidence in The Sims 3 (which at least is a top ten game). But the point is that I think I almost didn’t catch my mistake before it was too late, and this kind of error may be common.
I was confident in my incorrect computer game answer because I had recently read this Wikipedia page List of best-selling video games remembered the answer and unthinkingly assumed that “video games” was the same as “computer games”.
The correct answer is Tetris. The question should have been what is the best selling personal computer game of all time? Mobile phones are technically computers too. I’m not sure how much difference that would have made.
I interpreted the question to include mobile devices and answered Tetris with high confidence.
It would be interesting to see the results of the question if we accepted either Tetris or Minecraft as the correct answer, since both are correct depending on whether or not “computer” was meant to mean “IBM PC Compatible” or “video game playing platform”
On the computer game question: Isn’t there an implicit “X is true and X will be marked correct by the rater”? You’d hope these two are clearly aligned, but if you’ve taken many real-world quizzes, you’ll recognize the difference.
I think the computer games question has to do with tribal identity-people who love a particularly well known game might be more inclined to list it as being the best seller ever and put down higher confidence because they love it so much.
Kind of like owners of Playstations and Xboxs will debate the superiority of their technical specs regardless of whether they’re superior or not.
I think the computer games result has to do with it being a bad question. There are many legitimate answers depending on how you interpret the question, including my answer that Minesweeper sells as a bundle with Windows and thus has probably sold more copies than anything else.
Is it really a “bad question”? Shouldn’t a good calibrator be able to account for model error?
Depends on whether you consider “being able to comprehensively understand questions that may be misleading” to be a subset of calibration skills.
Good point, I hadn’t thought of that.