Great works, folks. This further highlights a challenge that wasn’t obvious to me when I first began to study SAEs — which features are learned is just super contingent on the SAE size, sparsity, and training data. Ablations like this one are important.
Great works, folks. This further highlights a challenge that wasn’t obvious to me when I first began to study SAEs — which features are learned is just super contingent on the SAE size, sparsity, and training data. Ablations like this one are important.