The point non-programmers tend to miss here is that lack of testing doesn’t just mean the model is a a little off. It means the model has no connection at all to reality, and either outputs garbage or echoes whatever result the programmer told it to give. Any programmer who claims such a model means something is committing fraud, plain and simple.
This really is a pretty un-bayesian way of thinking—the idea that we should totally ignore incomplete evidence. And by extension that we should chose to believe an alternative hypothesis (″no nuclear winter’) with even less evidence merely because it is assumed for unstated reasons to be the ‘default belief’.
An uncalibrated sim will typically give crazy results like ‘increasing atmospheric CO2 by 1% raises surface temperatures by 300 degrees’ or ‘one large forest fire will trigger a permanent ice age’. If you see an uncalibrated sim giving results that seem even vaguely plausible, this means the programmer has tinkered with its internal mechanisms to make it give those results. Doing that is basically equivalent to just typing up the desired output by hand—it provides evidence about the beliefs of the programmer, but nothing else.
This really is a pretty un-bayesian way of thinking—the idea that we should totally ignore incomplete evidence. And by extension that we should chose to believe an alternative hypothesis (″no nuclear winter’) with even less evidence merely because it is assumed for unstated reasons to be the ‘default belief’.
An uncalibrated sim will typically give crazy results like ‘increasing atmospheric CO2 by 1% raises surface temperatures by 300 degrees’ or ‘one large forest fire will trigger a permanent ice age’. If you see an uncalibrated sim giving results that seem even vaguely plausible, this means the programmer has tinkered with its internal mechanisms to make it give those results. Doing that is basically equivalent to just typing up the desired output by hand—it provides evidence about the beliefs of the programmer, but nothing else.