large part of the problem is that all the lessons of Traditional Rationality teach to guard against actually arriving to conclusions before amassing what I think one Sequence post called “mountains of evidence”.
Except for scientific research, which will happily accept p < 0.05 to publish the most improbable claims.
And even with more than 5 sigma people will be like ‘we probably screwed up somewhere’ when the claim is sufficiently improbable, see e.g. the last paragraph before the acknowledgements in arXiv:1109.4897v1.
Except for scientific research, which will happily accept p < 0.05 to publish the most improbable claims.
No, “real science” requires more evidence than that − 5 sigma in HEP. p < 0.05 is the preserve of “soft science”.
And even with more than 5 sigma people will be like ‘we probably screwed up somewhere’ when the claim is sufficiently improbable, see e.g. the last paragraph before the acknowledgements in arXiv:1109.4897v1.