The certainty level is effectively communicated via the significance level and p-value itself.
No. P-values are not equivalent when they are calculated using different statistics, or even the same statistic but a different sample size. On the latter point see Royall, 1986.
As I see it, the two say essentially the same thing; the frequentist is just being more specific than the Bayesian.
I’d say the frequentist is using Bayesian reasoning informally; Jaynes discusses this exact problem from a Bayesian perspective at the beginning of Chapter 5 of his magnum opus.
No. P-values are not equivalent when they are calculated using different statistics, or even the same statistic but a different sample size. On the latter point see Royall, 1986.
Sorry. You are quite right, and I was sloppy. I had in mind the implicit idea that holding the choices of statistical test and data collection procedure constant, different p-values suggest how strongly one should reject the null hypothesis, and I should have made that explicit. It is absolutely true that if I just ask someone, “Test A gave me p = 0.008 and Test B gave me p = 0.4, which test’s null hypothesis is worse off?”, the correct answer is “how should I know?”
I’d say the frequentist is using Bayesian reasoning informally; Jaynes discusses this exact problem from a Bayesian perspective at the beginning of Chapter 5 of his magnum opus.
Yep. I think this is an example of the frequentist encapsulating what a Bayesian would call priors in their sampling assumptions.
No. P-values are not equivalent when they are calculated using different statistics, or even the same statistic but a different sample size. On the latter point see Royall, 1986.
I’d say the frequentist is using Bayesian reasoning informally; Jaynes discusses this exact problem from a Bayesian perspective at the beginning of Chapter 5 of his magnum opus.
Sorry. You are quite right, and I was sloppy. I had in mind the implicit idea that holding the choices of statistical test and data collection procedure constant, different p-values suggest how strongly one should reject the null hypothesis, and I should have made that explicit. It is absolutely true that if I just ask someone, “Test A gave me p = 0.008 and Test B gave me p = 0.4, which test’s null hypothesis is worse off?”, the correct answer is “how should I know?”
Yep. I think this is an example of the frequentist encapsulating what a Bayesian would call priors in their sampling assumptions.