Thanks for the quick clarification! I guess I am not following you. If you want to blind yourself, you can just do it—you don’t need to modify the estimator in any way, just write the computer program implementing your estimator in such a way that you don’t see the answer. This issue seems to be completely orthogonal to both causal inference and estimation. (???)
If you use the propensity score matching method, you begin by estimating the propensity score, then you match on the propensity score to create exposed and unexposed groups within levels of the propensity score. After you create those groups, there is a step where you can look at the matched groups without the outcome data, and assess whether you have achieved balance on the baseline covariates. If I understand Rubin’s students correctly, they see this as a major advantage of the estimation method.
You can obviously blind yourself to the outcome using any estimation method, but I am not sure if there is a step in the process where you look at the data without the outcome to evaluate how confident you are in your work.
In order for the estimand of the propensity score method to be unbiased, the following is sufficient:
(a) SUTVA (this is untestable)
(b) Conditional ignorability (this is testable in principle, but only if we randomize the exposure A)
(c) The treatment assignment probability model (that is the model for p(A | C), where A is exposure and C is baseline covariates) must be correct.
It may be that the “balance property” tests a part of (b), but surely not all of it! That is, the arms might look balanced, but conditional ignorability might still not hold. We cannot test all the assumptions we need to draw causal conclusions from observational data using only observational data. Causal assumptions have to enter in somewhere!
I think I might be confused about why checking for balance without working out the effect is an advantage—but I will think about it, because I am not an expert on propensity score methods, so there is probably something I am missing.
Thanks for the quick clarification! I guess I am not following you. If you want to blind yourself, you can just do it—you don’t need to modify the estimator in any way, just write the computer program implementing your estimator in such a way that you don’t see the answer. This issue seems to be completely orthogonal to both causal inference and estimation. (???)
Am I missing something?
If you use the propensity score matching method, you begin by estimating the propensity score, then you match on the propensity score to create exposed and unexposed groups within levels of the propensity score. After you create those groups, there is a step where you can look at the matched groups without the outcome data, and assess whether you have achieved balance on the baseline covariates. If I understand Rubin’s students correctly, they see this as a major advantage of the estimation method.
You can obviously blind yourself to the outcome using any estimation method, but I am not sure if there is a step in the process where you look at the data without the outcome to evaluate how confident you are in your work.
In order for the estimand of the propensity score method to be unbiased, the following is sufficient:
(a) SUTVA (this is untestable)
(b) Conditional ignorability (this is testable in principle, but only if we randomize the exposure A)
(c) The treatment assignment probability model (that is the model for p(A | C), where A is exposure and C is baseline covariates) must be correct.
It may be that the “balance property” tests a part of (b), but surely not all of it! That is, the arms might look balanced, but conditional ignorability might still not hold. We cannot test all the assumptions we need to draw causal conclusions from observational data using only observational data. Causal assumptions have to enter in somewhere!
I think I might be confused about why checking for balance without working out the effect is an advantage—but I will think about it, because I am not an expert on propensity score methods, so there is probably something I am missing.