Robert_AIZI comments on Comparing Anthropic’s Dictionary Learning to Ours

[Robert_AIZI]

This is something we’re planning to look into! From the paper:

Future efforts could also try to improve feature dictionary discovery by incorporating information about the weights of the model or dictionary features found in adjacent layers into the training process.

Exactly how to use them is something we’re still working on...