Erm… let me be Brennan and go with the “obvious”. Find problems whose solutions are known in some field but not widely, provide the initial data and results of additional experiments on request (with “too expensive to perform” being a possible result). Then have two measures:
1)Someone who is _also not an expert_ checks solutions for, well, everything you discuss here. Biases, effort, mysterious answers—you name it. (For effort, you might need to register when every thought was written, not just what it was.)
2)An expert checks the dataset used—what of the really conducted experiments students failed to request and which of them were actually useful.
Erm… let me be Brennan and go with the “obvious”. Find problems whose solutions are known in some field but not widely, provide the initial data and results of additional experiments on request (with “too expensive to perform” being a possible result). Then have two measures:
1)Someone who is _also not an expert_ checks solutions for, well, everything you discuss here. Biases, effort, mysterious answers—you name it. (For effort, you might need to register when every thought was written, not just what it was.)
2)An expert checks the dataset used—what of the really conducted experiments students failed to request and which of them were actually useful.