They key point is that when you do the p value test you are determining p(data | null_hyp). This is certainly useful to calculate, but doesn’t tell you the whole story about whether your data support any particular non-null hypotheses.
Chapter 17 of E.T. Jaynes’ book provides a lively discussion of the limitations of traditional hypothesis testing, and is accessible enough that you can dive into it without having worked through the rest of the book.
The Cohen article cited below is nice but it’s important to note it doesn’t completely reject the use of null hypotheses or p-values:
.. null hypothesis testing complete with power analysis can be useful if we abandon the rejection of point nil hypotheses and use instead “good-enough” range null hypotheses
I’m not sure it’s so clear cut.
They key point is that when you do the p value test you are determining p(data | null_hyp). This is certainly useful to calculate, but doesn’t tell you the whole story about whether your data support any particular non-null hypotheses.
Chapter 17 of E.T. Jaynes’ book provides a lively discussion of the limitations of traditional hypothesis testing, and is accessible enough that you can dive into it without having worked through the rest of the book.
The Cohen article cited below is nice but it’s important to note it doesn’t completely reject the use of null hypotheses or p-values: