Suppose you have three hypotheses:
(1) It’s better to email in the morning
(2) It’s better to email in the evening
(3) They’re equally good
Why do you care about (3)? If you’re just deciding whether to email in the morning or evening, (3) is irrelevant to ranking those two options.
The full-fledged Bayesian approach would be to identify the hypotheses (I’ve simplified it by reducing it down to just three), decide what your priors are, calculate the probability of seeing the data under each of the hypotheses, and then combing that data according to the Bayesian formula to find the posterior probability. However, you don’t have to run through the math to see that if your prior for (1) and (2) are equal, and the sample is skewed towards evening, then the posterior for (2) will be larger than the posterior for (1).
The only time you’d actually have to run through the math is if your priors weren’t equal, and you’re trying to decide whether the additional data is enough to overcome the difference in the priors, or if you have some consideration other than just choosing between morning or evening (for instance, you might find it more convenient to just email when you first have something to email about, in which case you’re choosing between “email in morning”, “email in evening” and “email whenever it’s convenient to me”).
“Statistical significance” is just a shorthand to avoid having to actually doing a Bayesian calculation. For instance, suppose we’re trying to decide whether a study showing that a drug is effective is statistically significant. If the only two choices were “take the drug” and “don’t take the drug”, and we were truly indifferent between those two options, the issue of significance wouldn’t even matter. We should just take the drug. The reason we care about whether the test is significant is because we aren’t indifferent to the two choices (we have a bias towards the status quo of not taking the drug, making the drug would cost money, there are probably going to be side effects of the drug, etc.) and there are other options (take another drug, have more drug trials, etc.) When a level of statistical significance is chosen, an implicit statement is being made about how much weight is being given towards the status quo.
Suppose you have three hypotheses: (1) It’s better to email in the morning (2) It’s better to email in the evening (3) They’re equally good
Why do you care about (3)? If you’re just deciding whether to email in the morning or evening, (3) is irrelevant to ranking those two options.
The full-fledged Bayesian approach would be to identify the hypotheses (I’ve simplified it by reducing it down to just three), decide what your priors are, calculate the probability of seeing the data under each of the hypotheses, and then combing that data according to the Bayesian formula to find the posterior probability. However, you don’t have to run through the math to see that if your prior for (1) and (2) are equal, and the sample is skewed towards evening, then the posterior for (2) will be larger than the posterior for (1).
The only time you’d actually have to run through the math is if your priors weren’t equal, and you’re trying to decide whether the additional data is enough to overcome the difference in the priors, or if you have some consideration other than just choosing between morning or evening (for instance, you might find it more convenient to just email when you first have something to email about, in which case you’re choosing between “email in morning”, “email in evening” and “email whenever it’s convenient to me”).
“Statistical significance” is just a shorthand to avoid having to actually doing a Bayesian calculation. For instance, suppose we’re trying to decide whether a study showing that a drug is effective is statistically significant. If the only two choices were “take the drug” and “don’t take the drug”, and we were truly indifferent between those two options, the issue of significance wouldn’t even matter. We should just take the drug. The reason we care about whether the test is significant is because we aren’t indifferent to the two choices (we have a bias towards the status quo of not taking the drug, making the drug would cost money, there are probably going to be side effects of the drug, etc.) and there are other options (take another drug, have more drug trials, etc.) When a level of statistical significance is chosen, an implicit statement is being made about how much weight is being given towards the status quo.