A single study is never definitive, because science involves reproducible results based on empirical evidence.
That simply doesn’t follow: why does involving reproducible results imply not being definitive?
Empirical results are never ‘definitive’ as in being 100.0% certain, but we can get very close. Whether this is done in a single study or with multiple studies doesn’t matter at all. In practice there are good reasons to want multiple studies, but they have more to do with questions not addressed in a single study, trustworthiness of the authors, etc.
On the other hand, encountering a single incorrect premise or step means that the conclusion has zero utility
Even wrong mathematical proofs have a non-zero utility, because they often lead to new insights. For example, if only the last of 100 steps is wrong, then you are 99 steps closer to some goal.
A single study can’t get close to 100% certainty, because that’s just not how science works. If you look at all the studies that were true with 95% certainty, you’ll find that well over 5% have found conclusions now believed to be false. There are issues of trust, issues of data collection errors, issues of statistical evaluation, the fact that scientific methods are designed under the assumption that studies will be repeated, etc.
The steps within unsound mathematical proofs may be valuable, but their conclusions are not.
A single study can’t get close to 100% certainty, because that’s just not how science works. … the fact that scientific methods are designed under the assumption that studies will be repeated, etc.
The current scientific method is in no way ideal. If a study were properly Bayesian, then you should be able to confidently learn from its results. That still leaves issues of trust and the possibility of human error, but there might also be ways to combat those. But in a human society, repeating studies is perhaps the best thing one can hope for.
The steps within unsound mathematical proofs may be valuable, but their conclusions are not.
Agreed. That is the one part of an unsound proof that is useless.
Can you describe a better, more Bayesian scientific method? The main way I would change it is to increase the number of studies that are repeated, to improve the accuracy of our knowledge. How would you propose to improve our confidence other than by showing that an experiment has reproducible results?
That simply doesn’t follow: why does involving reproducible results imply not being definitive?
Empirical results are never ‘definitive’ as in being 100.0% certain, but we can get very close. Whether this is done in a single study or with multiple studies doesn’t matter at all. In practice there are good reasons to want multiple studies, but they have more to do with questions not addressed in a single study, trustworthiness of the authors, etc.
Even wrong mathematical proofs have a non-zero utility, because they often lead to new insights. For example, if only the last of 100 steps is wrong, then you are 99 steps closer to some goal.
A single study can’t get close to 100% certainty, because that’s just not how science works. If you look at all the studies that were true with 95% certainty, you’ll find that well over 5% have found conclusions now believed to be false. There are issues of trust, issues of data collection errors, issues of statistical evaluation, the fact that scientific methods are designed under the assumption that studies will be repeated, etc.
The steps within unsound mathematical proofs may be valuable, but their conclusions are not.
The current scientific method is in no way ideal. If a study were properly Bayesian, then you should be able to confidently learn from its results. That still leaves issues of trust and the possibility of human error, but there might also be ways to combat those. But in a human society, repeating studies is perhaps the best thing one can hope for.
Agreed. That is the one part of an unsound proof that is useless.
Can you describe a better, more Bayesian scientific method? The main way I would change it is to increase the number of studies that are repeated, to improve the accuracy of our knowledge. How would you propose to improve our confidence other than by showing that an experiment has reproducible results?