There is a true result of the run, stored in the variable resultForX. While I’m developing my code, I don’t want to know that true value, because of the surprisingness bias as outlined in the post. I do however want to be able to compare results between test runs. Thus I add a random value, blindValueX, which I do not know; I only know the random seed that produces it. I never print the true result until I’ve finalised the code and done all my testing for systematic errors.
It is, at any rate, quite common with particle physics, although not every analysis uses it. I can’t speak to other fields.
Hmm. I wonder if this would make a top-level post, with some example plots and more in-depth description? Practical methods used in science for avoiding bias, 101.
There is a true result of the run, stored in the variable resultForX. While I’m developing my code, I don’t want to know that true value, because of the surprisingness bias as outlined in the post. I do however want to be able to compare results between test runs. Thus I add a random value, blindValueX, which I do not know; I only know the random seed that produces it. I never print the true result until I’ve finalised the code and done all my testing for systematic errors.
Okay; I see. Is that a common practice? I’d never heard of it before.
It is, at any rate, quite common with particle physics, although not every analysis uses it. I can’t speak to other fields.
Hmm. I wonder if this would make a top-level post, with some example plots and more in-depth description? Practical methods used in science for avoiding bias, 101.