So, you’ve thrown in a link to a paper which you clearly didn’t even glance at because it has nothing to do with performing reproducible measurements.
Here is the abstract for you:
“The previous articles in this special section make the case for the importance of evaluating the
clinical significance of therapeutic change, present key measures and innovative ways in which
they are applied, and more generally provide important guidelines for evaluating therapeutic
change. Fundamental issues raised by the concept of clinical significance and the methods
discussed in the previous articles serve as the basis of the present comments. Salient among
these issues are ambiguities regarding the meaning of current measures of clinical significance,
the importance of relating assessment of clinical significance to the goals of therapy, and
evaluation of the construct(s) that clinical significance reflects. Research directions that are
discussed include developing a typology of therapy goals, evaluating cutoff scores and
thresholds for clinical significance, and attending to social as well as clinical impact of treatment.”
Do note the part that mentions ambiguities regarding the meaning of current measures.
I did skim it and it adresses all the relevant aspects. It is indeed the first hit that came up but it does show a very rigorous and scientific treatment of the topic. It is also balanced in so far as it separated out statistical measures from other valuations (to a avoid calling it “bayesian priors” which he does’t claim):
Apart from reliability of change or group differences (e.g., statistical
significance) and the magnitude of experimental effects (e.g., effect size or correlation), the importance of the
change and the impact on client functioning add critical dimensions. Treatments that produce reliable effects
may be quite different in their impact on client functioning, and clinical significance brings this issue to light.
It tells me that Kazdin knows quite well “how to perform reproducible measurements”, not how these measurements are carried out in particular. It seems that there are more papers out there that actually do this.
So, you’ve thrown in a link to a paper which you clearly didn’t even glance at because it has nothing to do with performing reproducible measurements.
Here is the abstract for you:
“The previous articles in this special section make the case for the importance of evaluating the clinical significance of therapeutic change, present key measures and innovative ways in which they are applied, and more generally provide important guidelines for evaluating therapeutic change. Fundamental issues raised by the concept of clinical significance and the methods discussed in the previous articles serve as the basis of the present comments. Salient among these issues are ambiguities regarding the meaning of current measures of clinical significance, the importance of relating assessment of clinical significance to the goals of therapy, and evaluation of the construct(s) that clinical significance reflects. Research directions that are discussed include developing a typology of therapy goals, evaluating cutoff scores and thresholds for clinical significance, and attending to social as well as clinical impact of treatment.”
Do note the part that mentions ambiguities regarding the meaning of current measures.
I did skim it and it adresses all the relevant aspects. It is indeed the first hit that came up but it does show a very rigorous and scientific treatment of the topic. It is also balanced in so far as it separated out statistical measures from other valuations (to a avoid calling it “bayesian priors” which he does’t claim):
So, do show where does this particular paper tell you how to, in your words, “perform reproducible measurements”.
Your quote talks about interpretation of measurements—it says nothing about how to make sure the measurement itself is reliable and reproducible.
It tells me that Kazdin knows quite well “how to perform reproducible measurements”, not how these measurements are carried out in particular. It seems that there are more papers out there that actually do this.