“Two medical researchers use the same treatment independently, in different hospitals. Neither would stoop to falsifying the data, but one had decided beforehand that because of finite resources he would stop after treating N=100 patients, however many cures were observed by then. The other had staked his reputation on the efficacy of the treatment, and decided he would not stop until he had data indicating a rate of cures definitely greater than 60%, however many patients that might require. But in fact, both stopped with exactly the same data: n = 100 [patients], r = 70 [cures]. Should we then draw different conclusions from their experiments?” (Presumably the two control groups also had equal results.)
It both annoys and amuses me greatly that neither in the original post nor in the ensuing discussion there was a hint of a suggestion to actually perform the experiment by generating and analyzing multiple runs of data over this, rather small, parameter space. How long can it take to write a simulation in the language of your choice and let it run for a bit?
No logic, not even Bayesian logic, is a substitute for experimental verification.
Well, in this case you shouldn’t need the experiment if you can just calculate the results mathematically. Its like demanding that even though we have a proof of Fermat’s Last Theorem we should still pick random large integers and check it. Doing maths by the scientific method is somewhat ridiculous.
Well, in this case you shouldn’t need the experiment if you can just calculate the results mathematically.
How do you know you did not forget some salient features of the real-life problem when constructing your proof? There is a good chance that modeling it would expose a previously missed angle of the problem.
How do you know you did not forget some salient features of the real-life problem when constructing your proof?
I fully agree that in the case of applying a mathematical model to the real world, it is worthwhile to test the predictions in case there is a false hidden assumption. However, what we are talking about here is applying a mathematical model to a computer simulation, any false assumptions that have come in will have done so in the step from real world to computer simulation rather than from computer simulation to mathematical analysis, the only assumption made in the analysis is that your computer works and your code is not buggy.
There is a good chance that modeling it would expose a previously missed angle of the problem.
This is false, the mathematical analysis is a complete solution of the problem as stated. Arguing that a salient feature may have been missed is like arguing the same for any mathematical proof, i.e. silly.
It both annoys and amuses me greatly that neither in the original post nor in the ensuing discussion there was a hint of a suggestion to actually perform the experiment by generating and analyzing multiple runs of data over this, rather small, parameter space. How long can it take to write a simulation in the language of your choice and let it run for a bit?
No logic, not even Bayesian logic, is a substitute for experimental verification.
Well, in this case you shouldn’t need the experiment if you can just calculate the results mathematically. Its like demanding that even though we have a proof of Fermat’s Last Theorem we should still pick random large integers and check it. Doing maths by the scientific method is somewhat ridiculous.
How do you know you did not forget some salient features of the real-life problem when constructing your proof? There is a good chance that modeling it would expose a previously missed angle of the problem.
I fully agree that in the case of applying a mathematical model to the real world, it is worthwhile to test the predictions in case there is a false hidden assumption. However, what we are talking about here is applying a mathematical model to a computer simulation, any false assumptions that have come in will have done so in the step from real world to computer simulation rather than from computer simulation to mathematical analysis, the only assumption made in the analysis is that your computer works and your code is not buggy.
This is false, the mathematical analysis is a complete solution of the problem as stated. Arguing that a salient feature may have been missed is like arguing the same for any mathematical proof, i.e. silly.
Well, how about doing it yourself and posting the results?