The null hypothesis in most papers is of the form “some real-valued random variable is equal to zero”. This could be an effect size, a regression coefficient, a vector of regression coefficients, et cetera.
If that’s your null hypothesis then the null hypothesis actually being true is an event of zero probability, in the sense that if your study had sufficient statistical power it would pick up on a tiny signal that would make the variable under consideration (statistically) significantly different from zero. If you believe there are no real-valued variables in the real world then it’s merely an event of probability ε>0 where ε is a tiny real number, and this distinction doesn’t matter for my purposes.
Incidentally, I think this is actually what happened with Bem’s parapsychology studies: his methodology was sound and he indeed picked up a small signal, but the signal was of experimenter effects and other problems in the experimental design rather than of paranormal phenomena. My claim is that no matter what you’re investigating, a sufficiently powerful study will always manage to pick up on such a small signal.
The point is that the null hypothesis being false doesn’t tell you much without a useful alternative hypothesis at hand. If someone tells you “populations 1 and 2 have different sample means”, the predictive value of that precise claim is nil.
The null hypothesis in most papers is of the form “some real-valued random variable is equal to zero”. This could be an effect size, a regression coefficient, a vector of regression coefficients, et cetera.
If that’s your null hypothesis then the null hypothesis actually being true is an event of zero probability, in the sense that if your study had sufficient statistical power it would pick up on a tiny signal that would make the variable under consideration (statistically) significantly different from zero. If you believe there are no real-valued variables in the real world then it’s merely an event of probability ε>0 where ε is a tiny real number, and this distinction doesn’t matter for my purposes.
Incidentally, I think this is actually what happened with Bem’s parapsychology studies: his methodology was sound and he indeed picked up a small signal, but the signal was of experimenter effects and other problems in the experimental design rather than of paranormal phenomena. My claim is that no matter what you’re investigating, a sufficiently powerful study will always manage to pick up on such a small signal.
The point is that the null hypothesis being false doesn’t tell you much without a useful alternative hypothesis at hand. If someone tells you “populations 1 and 2 have different sample means”, the predictive value of that precise claim is nil.
I understand much better now what you were saying. Thanks for clarifying.