What you’ve described is the “statistical hypothesis testing” technique, and yes, you’ve got it right. The only reason it functions at all is that by and large, people who use it aren’t stupid, and they know that they have to submit it to peer review to other people who aren’t stupid. Nevertheless, a lot of crap gets through, just because the approach is so wrong-headed. ETA: Oops! I left an important detail out of this response.
There are other techniques for frequentist statistics, e.g., unbiased estimators, minimum mean squared error estimators, method of moments, robust estimators, confidence intervals, confidence distributions, maximum likelihood, profile likelihood, empirical likelihood, empirical Bayes, estimating equations, PAC learning, etc., etc., ad nauseum.
What you’ve described is the “statistical hypothesis testing” technique, and yes, you’ve got it right. The only reason it functions at all is that by and large, people who use it aren’t stupid, and they know that they have to submit it to peer review to other people who aren’t stupid. Nevertheless, a lot of crap gets through, just because the approach is so wrong-headed. ETA: Oops! I left an important detail out of this response.
There are other techniques for frequentist statistics, e.g., unbiased estimators, minimum mean squared error estimators, method of moments, robust estimators, confidence intervals, confidence distributions, maximum likelihood, profile likelihood, empirical likelihood, empirical Bayes, estimating equations, PAC learning, etc., etc., ad nauseum.