If my estimate is 1000, and someone else’s is 300, that’s too big a discrepancy to explain by minor variations. It casts doubt on the assumption of identical thermometers. Assuming that I only have the other people’s estimates, and there’s no opportunity for discussion, I’ll search for reasons why we might have come up with completely different answers, but if I find no error in my own, I’ll discard all such outliers.
What if everyone else’s estimate is between 280 and 320? Do you discard your own estimate if it’s an outlier? Does the answer depend on whether you can find an error in your reasoning?
Maybe I’ve made an error no-one else made. Maybe everyone else made an error I didn’t make. (I have personally experienced this. I knew what error everyone else was making and stuck to my answer, which in the end turned out to be right.) The thing to do is to find out why the discrepancy happened; then I will know what to do about it.
In some situations this will not be possible. Then I will have to just make an optimal Bayesian calculation based on limited information, i.e. guess. But “optimal” no more implies “accurate” than “statistically significant” implies “important”.
What if everyone else’s estimate is between 280 and 320? Do you discard your own estimate if it’s an outlier? Does the answer depend on whether you can find an error in your reasoning?
Maybe I’ve made an error no-one else made. Maybe everyone else made an error I didn’t make. (I have personally experienced this. I knew what error everyone else was making and stuck to my answer, which in the end turned out to be right.) The thing to do is to find out why the discrepancy happened; then I will know what to do about it.
In some situations this will not be possible. Then I will have to just make an optimal Bayesian calculation based on limited information, i.e. guess. But “optimal” no more implies “accurate” than “statistically significant” implies “important”.