If you can answer correctly more statistical questions, how is that not being ‘better at statistical reasoning’?
Those are related abilities, but there’s being able to answer specific questions and then there’s being able to apply what you’ve learned more generally. For me, this particular question triggered more “aha! I’ve seen this one before!” than it triggered statistical thought. A correct answer to the question might give you a smidgen of information on whether the answerer can reason about statistics, but it probably gives you a lot more information about whether the answerer has seen the question before.
One superficial example of dealing with this problem is how, in my college discrete math class, the professor gave us a problem involving placing pigeons in holes, with the solution having nothing to do with the pigeonhole principle. Even better than obfuscating a problem, of course, is stating a novel one that exercises the skills you’re testing for.
Those are related abilities, but there’s being able to answer specific questions and then there’s being able to apply what you’ve learned more generally. For me, this particular question triggered more “aha! I’ve seen this one before!” than it triggered statistical thought. A correct answer to the question might give you a smidgen of information on whether the answerer can reason about statistics, but it probably gives you a lot more information about whether the answerer has seen the question before.
One superficial example of dealing with this problem is how, in my college discrete math class, the professor gave us a problem involving placing pigeons in holes, with the solution having nothing to do with the pigeonhole principle. Even better than obfuscating a problem, of course, is stating a novel one that exercises the skills you’re testing for.