When I first read the Sequences, one of the exciting posts was Causal Diagrams and Causal Models, which got me into the idea that one could discover the structure of causal networks using statistics. Another rationalist source which gave me similar hopes was Scott Alexander’s SSC Journal Club: Mental Disorders As Networks.
However, when I actually started applying these techniques to my own data, or to publicly available datasets, I often found that the techniques were unstable, and that one could easily infer plausible conditions where they would give the wrong results. It’s possible I had the wrong approach or something, but in my confusion I started reading up on what experts in causal inference had said, and I got the impression that they studied the problem for a while, initially finding some algorithms, but over time concluding that their algorithms didn’t work very well and that it is better to just have a human in the loop who specifies the causal networks.
So I mostly abandoned it, or saw it as a much more limited tool than I had before. But recently, John Wentworth argued that it was actually quite feasible in practice, so maybe I was too quick to abandon it. I would like to know—what are the best examples of this working well in practice? Or alternatively, did anyone else come to the same conclusions as I did?
[Question] Are there any good, easy-to-understand examples of cases where statistical causal network discovery worked well in practice?
When I first read the Sequences, one of the exciting posts was Causal Diagrams and Causal Models, which got me into the idea that one could discover the structure of causal networks using statistics. Another rationalist source which gave me similar hopes was Scott Alexander’s SSC Journal Club: Mental Disorders As Networks.
However, when I actually started applying these techniques to my own data, or to publicly available datasets, I often found that the techniques were unstable, and that one could easily infer plausible conditions where they would give the wrong results. It’s possible I had the wrong approach or something, but in my confusion I started reading up on what experts in causal inference had said, and I got the impression that they studied the problem for a while, initially finding some algorithms, but over time concluding that their algorithms didn’t work very well and that it is better to just have a human in the loop who specifies the causal networks.
So I mostly abandoned it, or saw it as a much more limited tool than I had before. But recently, John Wentworth argued that it was actually quite feasible in practice, so maybe I was too quick to abandon it. I would like to know—what are the best examples of this working well in practice? Or alternatively, did anyone else come to the same conclusions as I did?