(Pardon the goofy notation. Don’t want to deal with the LaTeX engine.)
The compound hypothesis is well-defined. Suppose that the baseline cure probability for a placebo is θ ∈ [0,1). Then hypotheses take the form H ⊂ [0,1], which have the interpretation that the cure rate for homeopathy is in H. The standing null hypothesis in this case is Hθ = { θ }. The alternative hypothesis that homeopathy works is H>θ = (θ,1] = { x : x > θ }. For any θ′ ∈ H>θ, we can construct a “one-in-a-million chance of being wrong” test for the simple hypothesis Hθ′ that homeopathy is effective with effect size exactly θ′. It is convenient that such tests work just as well for the hypothesis H≥θ′. However, we can’t construct a test for H>θ.
Bringing in falsifiability only confuses the issue. No clinical data exist that will strictly falsify any of the hypotheses considered above. On the other hand, rejecting Hθ′ seems like it should provide weak support for rejecting H>θ. My take on this is that since such a research program seems to work in practice, falsifiability doesn’t fully describe how science works in this case (see Popper vs. Kuhn, Lakatos, Feyerabend, etc.).
Clinical data still exists that would allow a strategy to stop doing more tests at specific cut off point as the payoff from the hypothesis being right is dependent to the size of the effect and there will be clinical data at some point where the integral of payoff over lost clinical effects is small enough. It just gets fairly annoying to calculate. . Taking the strategy will be similar to gambling decision.
I do agree that there is a place for occam’s razor here but there exist no formalism that actually lets you quantify this weak support. There’s the Solomonoff induction, which is un-computable and awesome for work like putting an upper bound on how good induction can (or rather, can’t) ever be.
It’s part of the hypothesis, without it the idea is not a defined hypothesis. See falsifiability.
(Pardon the goofy notation. Don’t want to deal with the LaTeX engine.)
The compound hypothesis is well-defined. Suppose that the baseline cure probability for a placebo is θ ∈ [0,1). Then hypotheses take the form H ⊂ [0,1], which have the interpretation that the cure rate for homeopathy is in H. The standing null hypothesis in this case is Hθ = { θ }. The alternative hypothesis that homeopathy works is H>θ = (θ,1] = { x : x > θ }. For any θ′ ∈ H>θ, we can construct a “one-in-a-million chance of being wrong” test for the simple hypothesis Hθ′ that homeopathy is effective with effect size exactly θ′. It is convenient that such tests work just as well for the hypothesis H≥θ′. However, we can’t construct a test for H>θ.
Bringing in falsifiability only confuses the issue. No clinical data exist that will strictly falsify any of the hypotheses considered above. On the other hand, rejecting Hθ′ seems like it should provide weak support for rejecting H>θ. My take on this is that since such a research program seems to work in practice, falsifiability doesn’t fully describe how science works in this case (see Popper vs. Kuhn, Lakatos, Feyerabend, etc.).
Clinical data still exists that would allow a strategy to stop doing more tests at specific cut off point as the payoff from the hypothesis being right is dependent to the size of the effect and there will be clinical data at some point where the integral of payoff over lost clinical effects is small enough. It just gets fairly annoying to calculate. . Taking the strategy will be similar to gambling decision.
I do agree that there is a place for occam’s razor here but there exist no formalism that actually lets you quantify this weak support. There’s the Solomonoff induction, which is un-computable and awesome for work like putting an upper bound on how good induction can (or rather, can’t) ever be.