It seems like (unless I just haven’t discovered it yet) there’s a sore need for a framework for causal model comparison, analogous to Bayesian model comparison. If you read Pearl (and his students), they rightfully point out that you can’t get causal claims without causal assumptions but don’t talk much about how you actually formulate the causal model in the first place (“domain knowledge”). As a result, if you look at the literature, researchers mostly seem to use a small set of causal models that may or may not describe phenomena, e.g. the classic “instrumental variable” graph, for inference.
I view this as analogous to selecting a prior in applied Bayesian modeling. However, there there’s a nice set of tools for comparing how likely different models are, whereas I’m not aware of any such thing in the causal inference world. There’s something called “sensitivity analysis” but that’s about how much deviation from your assumptions affects your conclusions.
It seems like (unless I just haven’t discovered it yet) there’s a sore need for a framework for causal model comparison, analogous to Bayesian model comparison. If you read Pearl (and his students), they rightfully point out that you can’t get causal claims without causal assumptions but don’t talk much about how you actually formulate the causal model in the first place (“domain knowledge”). As a result, if you look at the literature, researchers mostly seem to use a small set of causal models that may or may not describe phenomena, e.g. the classic “instrumental variable” graph, for inference.
I view this as analogous to selecting a prior in applied Bayesian modeling. However, there there’s a nice set of tools for comparing how likely different models are, whereas I’m not aware of any such thing in the causal inference world. There’s something called “sensitivity analysis” but that’s about how much deviation from your assumptions affects your conclusions.
I forgot to include the disclaimer besides statistical independence tests, which can invalidate graphs but are difficult in practice.