Anish Mudide comments on Efficient Dictionary Learning with Switch Sparse Autoencoders

[Anish Mudide]

Yes, you can train a Switch SAE in any scenario where you can train a standard SAE. @hugofry has a nice blog post on training SAEs for ViT.