I would be very surprised if this was not the case. Different fields already use different cutoffs for statistical-significance (you might get away with p<0.05 in psychology, but particle physics likes its five-sigmas, and in genomics the cutoff will be hundreds or thousands of times smaller and vary heavily based on what exactly you’re analyzing) and likewise have different expectations for effect sizes (psychology expects large effects, medicine expects medium effects, and genomics expects very small effects; eg for genetic influence on IQ, any claim of a allele with an effect larger than d=0.06 should be greeted with surprise and alarm).
The existing defaults aren’t usually well-justified: for example, why does psychology use 0.05 rather than 0.10 or 0.01?
This is a good point, and leads to what might be an interesting use of the experimental approach of linking correlations to causation: gauging whether the heuristics currently in use in a field are at a suitable level/reflect the degree to which correlation is evidence for causation.
If you were to find, for example, that physics is churning out huge sigmas where it doesn’t really need to, or psychology really really needs to up its standards of evidence (not that that in itself would be a surprising result), those could be very interesting results.
Of course, to run these experiments you need large samples of well-researched correlations you can easily and objectively test for causality, from all the fields you’re looking at, which is no small requirement.
This is a good point, and leads to what might be an interesting use of the experimental approach of linking correlations to causation: gauging whether the heuristics currently in use in a field are at a suitable level/reflect the degree to which correlation is evidence for causation.
If you were to find, for example, that physics is churning out huge sigmas where it doesn’t really need to, or psychology really really needs to up its standards of evidence (not that that in itself would be a surprising result), those could be very interesting results.
Of course, to run these experiments you need large samples of well-researched correlations you can easily and objectively test for causality, from all the fields you’re looking at, which is no small requirement.