Um, I don’t think the null hypothesis is usually phrased as, “There is no effect and our data wasn’t unusual” and then you conclude “our data was unusual, rather than there being no effect” when you get data with probability < .05 if the Sun hasn’t exploded. This is not a fair steelmanning.
I don’t follow. The null hypothesis can be phrased in all sorts of ways based on what you want to test—there there’s no effect, that the effect between two groups (eg. a new drug and an old drug) is the same etc.
then you conclude “our data was unusual, rather than there being no effect” when you get data with probability < .05 if the Sun hasn’t exploded.
I don’t know that my frequentist example does conclude the ‘data was unusual’ rather than ‘there was an effect’. I am not sure how a frequentist would break apart the disjunction, or indeed, if they even would without additional data and assumptions.
Null hypotheses are phrased in terms of presumed stochastic data-generating mechanism; they do not address the data directly. That said, you are right about the conclusion one is to draw from the test. Fisher himself phrased it as
Either the hypothesis is untrue, or the value of [the test statistic] has attained by chance an exceptionally high value. [emphasis in original as quoted here].
Um, I don’t think the null hypothesis is usually phrased as, “There is no effect and our data wasn’t unusual” and then you conclude “our data was unusual, rather than there being no effect” when you get data with probability < .05 if the Sun hasn’t exploded. This is not a fair steelmanning.
I don’t follow. The null hypothesis can be phrased in all sorts of ways based on what you want to test—there there’s no effect, that the effect between two groups (eg. a new drug and an old drug) is the same etc.
I don’t know that my frequentist example does conclude the ‘data was unusual’ rather than ‘there was an effect’. I am not sure how a frequentist would break apart the disjunction, or indeed, if they even would without additional data and assumptions.
Null hypotheses are phrased in terms of presumed stochastic data-generating mechanism; they do not address the data directly. That said, you are right about the conclusion one is to draw from the test. Fisher himself phrased it as