Unknown, I still find it difficult to accept that there should be literally zero modification. It’s important not just that n=100, but that n=100 random trials. Suppose both researchers reported 100% effectiveness with the same n, but researcher 2 threw out all the data points that suggested ineffectiveness? You still have an n=100 and a 100% effectiveness among that set, but any probability judgment that doesn’t account for the selective method of picking the population is inadequate. I would suggest that either less or a different kind of information is transmitted when the magnitude of n (i.e. the sample size) is not a precondition, and is crafted to enhance a certain outcome. It’s a subtler form of “throwing out the data” that doesn’t agree with you.
Why isn’t this just a much subtler version of the Monty Hall problem? It matters when and how the sample size was selected.
To put it in more concrete terms: if you assume, by hypothesis, that as n increases, the effectiveness percentage will fluctuate randomly due to statistical clustering, in the range between 54 and 61, then there is only an infinitesimal chance that, given truly infinite resources, researcher 2 from the original problem will not come up with the 60% figure, but the chance that researcher 1 will come up with the same figure is significantly less than 100%. If researcher 2 performs his experiment a large enough number of times, the distribution of his “n”s will be described by some kind of probability curve. And you will get 100% confirmation of the 60% figure. Whereas if you assign researcher 1 a priori the same distribution pattern of “n”s, his reported effectiveness percentage will be something like 57.5%. And you could come up with an infinite amount of evidence either way.
Now, of course, in real life the magnitude of n has some upper bound, so researcher 2 is playing a kind of martingale game with the facts, and I don’t know enough math to be sure that doesn’t precisely cancel out the difference, on average, between his data and that of researcher 1. But then again real life isn’t a thought experiment (at least not one we’re aware of), and researcher 2 is unlikely to abstain from some other subtle unconscious skewing of the data.
Unknown, I still find it difficult to accept that there should be literally zero modification. It’s important not just that n=100, but that n=100 random trials. Suppose both researchers reported 100% effectiveness with the same n, but researcher 2 threw out all the data points that suggested ineffectiveness? You still have an n=100 and a 100% effectiveness among that set, but any probability judgment that doesn’t account for the selective method of picking the population is inadequate. I would suggest that either less or a different kind of information is transmitted when the magnitude of n (i.e. the sample size) is not a precondition, and is crafted to enhance a certain outcome. It’s a subtler form of “throwing out the data” that doesn’t agree with you.
Why isn’t this just a much subtler version of the Monty Hall problem? It matters when and how the sample size was selected.
To put it in more concrete terms: if you assume, by hypothesis, that as n increases, the effectiveness percentage will fluctuate randomly due to statistical clustering, in the range between 54 and 61, then there is only an infinitesimal chance that, given truly infinite resources, researcher 2 from the original problem will not come up with the 60% figure, but the chance that researcher 1 will come up with the same figure is significantly less than 100%. If researcher 2 performs his experiment a large enough number of times, the distribution of his “n”s will be described by some kind of probability curve. And you will get 100% confirmation of the 60% figure. Whereas if you assign researcher 1 a priori the same distribution pattern of “n”s, his reported effectiveness percentage will be something like 57.5%. And you could come up with an infinite amount of evidence either way.
Now, of course, in real life the magnitude of n has some upper bound, so researcher 2 is playing a kind of martingale game with the facts, and I don’t know enough math to be sure that doesn’t precisely cancel out the difference, on average, between his data and that of researcher 1. But then again real life isn’t a thought experiment (at least not one we’re aware of), and researcher 2 is unlikely to abstain from some other subtle unconscious skewing of the data.