Neel Nanda comments on Sparse Autoencoders Work on Attention Layer Outputs

[Neel Nanda]

These models were not trained with dropout. Nice idea though!