in particular how much causal scrubbing can be turned into an exploratory tool to find circuits rather than just to verify them
I’d like to flag that this has been pretty easy to do—for instance, this process can look like resample ablating different nodes of the computational graph (eg each attention head/MLP), finding the nodes that when ablated most impact the model’s performance and are hence important, and then recursively searching for nodes that are relevant to the current set of important nodes by ablating nodes upstream to each important node.
I’d like to flag that this has been pretty easy to do—for instance, this process can look like resample ablating different nodes of the computational graph (eg each attention head/MLP), finding the nodes that when ablated most impact the model’s performance and are hence important, and then recursively searching for nodes that are relevant to the current set of important nodes by ablating nodes upstream to each important node.
Exciting! I look forward to the first “interesting circuit entirely derived by causal scrubbing” paper