Joe Kwon comments on Solving the Mechanistic Interpretability challenges: EIS VII Challenge 2

[Joe Kwon]

Super cool stuff. Minor question, what does “Fraction of MLP progress” mean? Are you scaling down the MLP output values that get added to the residual stream? Thanks!