Lucius Bushnaq comments on A List of 45+ Mech Interp Project Ideas from Apollo Research’s Interpretability Team

[Lucius Bushnaq]

A prior that doesn’t assume independence should give you a sparsity penalty that isn’t a sum of independent penalties for each activation.