tailcalled comments on Why I’m bearish on mechanistic interpretability: the shards are not in the network

[tailcalled]

As I mentioned in my other comment, SAEs finds features that correspond to abstract features of words and text. That’s not the same as finding features that correspond to reality.