When you have measured covariates, you can compare their sampled distribution to that of the population of interest.
That means that you need more data. Having a standard against which to train your model means that you need more than just the results of your measurement.
I was just contesting your statement as a universal one. For this poll, I agree you can’t really pursue the covariate strategy. However, I think you’re overstating challenge of getting more data and figuring out what to do with it.
For example, measuring BPD status is difficult. You can do it by conducting a psychological examination of your subjects (costly but accurate), you can do it by asking subjects to self-report on a four-level Likert-ish scale (cheap but inaccurate), or you could do countless other things along this tradeoff surface. On the other hand, measuring things like sex, age, level of education, etc. is easy. And even better, we have baseline levels of these covariates for communities like LessWrong, the United States, etc. with respect to which we might want to see if our sample is biased.
I was just contesting your statement as a universal one.
You argued against a more general statement than the one I made. But I did choose my words in a way that focused on drawing conclusions from the results and not results + comparison data.
That means that you need more data. Having a standard against which to train your model means that you need more than just the results of your measurement.
I was just contesting your statement as a universal one. For this poll, I agree you can’t really pursue the covariate strategy. However, I think you’re overstating challenge of getting more data and figuring out what to do with it.
For example, measuring BPD status is difficult. You can do it by conducting a psychological examination of your subjects (costly but accurate), you can do it by asking subjects to self-report on a four-level Likert-ish scale (cheap but inaccurate), or you could do countless other things along this tradeoff surface. On the other hand, measuring things like sex, age, level of education, etc. is easy. And even better, we have baseline levels of these covariates for communities like LessWrong, the United States, etc. with respect to which we might want to see if our sample is biased.
You argued against a more general statement than the one I made. But I did choose my words in a way that focused on drawing conclusions from the results and not results + comparison data.