The setup here implies a empirical (but conceptually tricky) research direction: try to take two different AIs trained to both do the same prediction task (e.g. predict next tokens of webtext) and try to correspond their internal structure in some way.
It’s a bit unclear to me what the desiderata for this research should be. I think we ideally want something like a “mechanistic correspondence”, something like a heuristic argument that the two models produce the same output distribution when given the same input.
Back when Redwood was working on model internals and interp, we were somewhat excited about trying to do something along these lines. Probably something trying to use automated methods to do a correspondence that seems accurate based on causal scrubbing.
(I haven’t engaged much with this post overall, I just thought this connection might be interesting.)
The setup here implies a empirical (but conceptually tricky) research direction: try to take two different AIs trained to both do the same prediction task (e.g. predict next tokens of webtext) and try to correspond their internal structure in some way.
It’s a bit unclear to me what the desiderata for this research should be. I think we ideally want something like a “mechanistic correspondence”, something like a heuristic argument that the two models produce the same output distribution when given the same input.
Back when Redwood was working on model internals and interp, we were somewhat excited about trying to do something along these lines. Probably something trying to use automated methods to do a correspondence that seems accurate based on causal scrubbing.
(I haven’t engaged much with this post overall, I just thought this connection might be interesting.)