No, it’s not fair. Given the setup, the null hypothesis would be, I think, ‘neither the Sun has exploded nor the dice come up 6’, and so when the detector goes off we reject the ‘neither x nor y’ in favor of ‘x or y’ - and I think the Bayesian would agree too that ‘either the Sun has exploded or the dice came up 6’!
Um, I don’t think the null hypothesis is usually phrased as, “There is no effect and our data wasn’t unusual” and then you conclude “our data was unusual, rather than there being no effect” when you get data with probability < .05 if the Sun hasn’t exploded. This is not a fair steelmanning.
I don’t follow. The null hypothesis can be phrased in all sorts of ways based on what you want to test—there there’s no effect, that the effect between two groups (eg. a new drug and an old drug) is the same etc.
then you conclude “our data was unusual, rather than there being no effect” when you get data with probability < .05 if the Sun hasn’t exploded.
I don’t know that my frequentist example does conclude the ‘data was unusual’ rather than ‘there was an effect’. I am not sure how a frequentist would break apart the disjunction, or indeed, if they even would without additional data and assumptions.
Null hypotheses are phrased in terms of presumed stochastic data-generating mechanism; they do not address the data directly. That said, you are right about the conclusion one is to draw from the test. Fisher himself phrased it as
Either the hypothesis is untrue, or the value of [the test statistic] has attained by chance an exceptionally high value. [emphasis in original as quoted here].
neither the Sun has exploded nor the dice come up 6
Given the statement of the problem, this null hypothesis is not at all probabilistic—we know it’s false using deduction! This is an awful strange thing for a hypothesis to be in a problem that’s supposed to be about probabilities.
Since probabilistic reasoning is a superset of deductive logic (pace our Saint Jaynes, RIP), it’s not a surprise if some formulations of some problems turn out that way.
probabilistic reasoning is a superset of deductive logic (pace our Saint Jaynes
Ah, you mean like in chapter 1 of his book? :P
Anyhow, I think this should be surprising. Deductive logic is all well and good, but merely exercising it, with no mention of probabilities, is not the characteristic behavior of something called an “interpretation of probability.” If I run a vaccine trial and none of the participants get infected, my deductive conclusion is “either the vaccine worked, or it didn’t and something else made none of the participants get infected—QED.” And then I would submit this to The Lancet, and the reviewers would write me polite letters saying “could you do some statistical analyses?”
And you might say ‘well, I don’t know what ‘something else’ is, I can’t define it as a limit of any frequency!′ At least, not with more info than is presented in a 3 panel comic. (‘I am pretty darn sure about that disjunction, though.’)
I think the null hypothesis is “the neutrino detector is lying” because the question we are most interested in is if it is correctly telling us the sun has gone nova. If H0 is the null hypothesis and u1 is the chance of a neutrino event and u2 is the odds of double sixes then H0 = µ1 - µ2. Since the odds of two die coming up sixes is vastly larger than the odds of the sun going nova in our lifetime the test is not fair.
Among candidate stars for going nova I would think you could treat it as a random event. But Sol is not a candidate and so doesn’t even make it into the sample set. So it’s a very badly constructed setup. It’s like looking for a needle in 200 million haystacks but restricting yourself only to those haystacks you already know it cannot be in. Or do I have that wrong.
No, it’s not fair. Given the setup, the null hypothesis would be, I think, ‘neither the Sun has exploded nor the dice come up 6’, and so when the detector goes off we reject the ‘neither x nor y’ in favor of ‘x or y’ - and I think the Bayesian would agree too that ‘either the Sun has exploded or the dice came up 6’!
Um, I don’t think the null hypothesis is usually phrased as, “There is no effect and our data wasn’t unusual” and then you conclude “our data was unusual, rather than there being no effect” when you get data with probability < .05 if the Sun hasn’t exploded. This is not a fair steelmanning.
I don’t follow. The null hypothesis can be phrased in all sorts of ways based on what you want to test—there there’s no effect, that the effect between two groups (eg. a new drug and an old drug) is the same etc.
I don’t know that my frequentist example does conclude the ‘data was unusual’ rather than ‘there was an effect’. I am not sure how a frequentist would break apart the disjunction, or indeed, if they even would without additional data and assumptions.
Null hypotheses are phrased in terms of presumed stochastic data-generating mechanism; they do not address the data directly. That said, you are right about the conclusion one is to draw from the test. Fisher himself phrased it as
Given the statement of the problem, this null hypothesis is not at all probabilistic—we know it’s false using deduction! This is an awful strange thing for a hypothesis to be in a problem that’s supposed to be about probabilities.
Since probabilistic reasoning is a superset of deductive logic (pace our Saint Jaynes, RIP), it’s not a surprise if some formulations of some problems turn out that way.
Ah, you mean like in chapter 1 of his book? :P
Anyhow, I think this should be surprising. Deductive logic is all well and good, but merely exercising it, with no mention of probabilities, is not the characteristic behavior of something called an “interpretation of probability.” If I run a vaccine trial and none of the participants get infected, my deductive conclusion is “either the vaccine worked, or it didn’t and something else made none of the participants get infected—QED.” And then I would submit this to The Lancet, and the reviewers would write me polite letters saying “could you do some statistical analyses?”
And you might say ‘well, I don’t know what ‘something else’ is, I can’t define it as a limit of any frequency!′ At least, not with more info than is presented in a 3 panel comic. (‘I am pretty darn sure about that disjunction, though.’)
“The machine has malfunctioned.”
Why, I deny that, for the machine worked precisely as XKCD said it did.
I think the null hypothesis is “the neutrino detector is lying” because the question we are most interested in is if it is correctly telling us the sun has gone nova. If H0 is the null hypothesis and u1 is the chance of a neutrino event and u2 is the odds of double sixes then H0 = µ1 - µ2. Since the odds of two die coming up sixes is vastly larger than the odds of the sun going nova in our lifetime the test is not fair.
I don’t think one would simply ignore the dice, and what data is the frequentist drawing upon in the comic which specifies the null?
How about “the probability of our sun going nova is zero and 36 times zero is still zero”?
Although… continuing with the XKCD theme if you divide by zero perhaps that would increase the odds. ;)
Since the sun going nova is not a random event, strict frequentists deny that there is a probability to associate with it.
Among candidate stars for going nova I would think you could treat it as a random event. But Sol is not a candidate and so doesn’t even make it into the sample set. So it’s a very badly constructed setup. It’s like looking for a needle in 200 million haystacks but restricting yourself only to those haystacks you already know it cannot be in. Or do I have that wrong.
I’m going to try the Socratic method...
Is a coin flip a random event?
taboo random event.
it’s deterministic, but you don’t know how it will come out.