ISO quality certification doesn’t look primarily at the results, but primarily at the process. If the process has a good argument or justification that it consistently produces high quality, then it is deemed to be compliant. For example “we measure performance in [this] way, the records are kept in [this] way, quality problems are addressed like [this], compliance is addressed like [such-and-so]”.
I can imagine a simple checklist for rationality, analogous to the software carpentry checklist.
Do you have a procedure for making decisions?
Is the procedure available at the times and locations that you make decisions?
How do you prevent yourself from making decisions without following this procedure?
If your procedure depends on calibration data, how do you guarantee the quality of your calibration data?
How does your procedure address (common rationality failure #1)?
et cetera
Sorry, it’s just a sketch of a checklist, not a real proposal, but I think you get the idea. Test the process, not the results. Of course, the process should describe how it tests the results.
Consult someone else who commented on this article. I didn’t have an idea for how to solve Dr. Hanson’s original question. I was trying to pull the question sideways.
ISO quality certification doesn’t look primarily at the results, but primarily at the process. If the process has a good argument or justification that it consistently produces high quality, then it is deemed to be compliant. For example “we measure performance in [this] way, the records are kept in [this] way, quality problems are addressed like [this], compliance is addressed like [such-and-so]”.
I can imagine a simple checklist for rationality, analogous to the software carpentry checklist.
Do you have a procedure for making decisions?
Is the procedure available at the times and locations that you make decisions?
How do you prevent yourself from making decisions without following this procedure?
If your procedure depends on calibration data, how do you guarantee the quality of your calibration data?
How does your procedure address (common rationality failure #1)?
et cetera
Sorry, it’s just a sketch of a checklist, not a real proposal, but I think you get the idea. Test the process, not the results. Of course, the process should describe how it tests the results.
How do you know whether the checklist actually works or if it’s just pointless drudgery?
Sorry, I said “Test the process, not the results”, which is a strictly wrong misstatement. It is over-strong in the manner of a slogan.
A more accurate statement would be “Focus primarily on testing process, and secondarily on testing results.”
Okay but how do you test the results?
Consult someone else who commented on this article. I didn’t have an idea for how to solve Dr. Hanson’s original question. I was trying to pull the question sideways.