I’m not sure this quite follows, as it isn’t clear that the 14% were guessing randomly… they might have been operating from some systematically wrong model. But I agree that it’s likely.
To be fair, I also often wonder whether the people who take these tests are just picking random answers because they aren’t at all invested in the test. Back when I was taking neuropsych evaluations after my stroke, there were a number of questions intended to detect malingering; I would like to believe that tests like those cited here have similar “checksum” questions built in and that reported results would take that into account, but in fact I don’t.
Yeah—good point. I did realize that people probably do operate from a systematically wrong model—why not? But I figured that there are probably at least as many operating from a systematically wrong model that just happens to give them the right answer. I figured that if you were just guessing, with minimal information or reasoning (as opposed to none), it would more likely be biased towards the right answer.
I’m not sure this quite follows, as it isn’t clear that the 14% were guessing randomly… they might have been operating from some systematically wrong model. But I agree that it’s likely.
To be fair, I also often wonder whether the people who take these tests are just picking random answers because they aren’t at all invested in the test. Back when I was taking neuropsych evaluations after my stroke, there were a number of questions intended to detect malingering; I would like to believe that tests like those cited here have similar “checksum” questions built in and that reported results would take that into account, but in fact I don’t.
Yeah—good point. I did realize that people probably do operate from a systematically wrong model—why not? But I figured that there are probably at least as many operating from a systematically wrong model that just happens to give them the right answer. I figured that if you were just guessing, with minimal information or reasoning (as opposed to none), it would more likely be biased towards the right answer.