How much scheming/deception can we catch with “super dumb mech interp”?
By “super dumb mech interp”, I mean something like:
Train a linear probe on a dataset of scary words: “hacking”, “deception”, “scheming”, “collusion”, “exploit”, etc.
Flag a model whenever its activations point in this direction and the task doesn’t require thinking about these concepts.
Like, does this capture 80% of the potential scheming, and we need “smart” mech interp to catch the other 20%? Or does this technique capture pretty much none of the in-the-wild scheming?
How much scheming/deception can we catch with “super dumb mech interp”?
By “super dumb mech interp”, I mean something like:
Train a linear probe on a dataset of scary words: “hacking”, “deception”, “scheming”, “collusion”, “exploit”, etc.
Flag a model whenever its activations point in this direction and the task doesn’t require thinking about these concepts.
Like, does this capture 80% of the potential scheming, and we need “smart” mech interp to catch the other 20%? Or does this technique capture pretty much none of the in-the-wild scheming?
Would appreciate any intuitions here. Thanks.