Excited by this direction! I think it would be nice to run your analysis on SAEs that are the same size but have different seeds (for dataset and parameter initialisation). It would be interesting to compare how the proportion and raw number of “new info features” and “similar info features” differ between same size SAEs and larger SAEs.
Excited by this direction! I think it would be nice to run your analysis on SAEs that are the same size but have different seeds (for dataset and parameter initialisation). It would be interesting to compare how the proportion and raw number of “new info features” and “similar info features” differ between same size SAEs and larger SAEs.