ntt123

• ntt123Jun 18, 2024, 1:38 AM

  3 points

  0

  in reply to: Logan Riggs’s comment on: Logit Prisms: De­com­pos­ing Trans­former Out­puts for Mechanis­tic Interpretability

  Thank you for the upvote! My main frustration with logit lens and tuned lens is that these methods are kind of ad hoc and do not reflect component contributions in a mathematically sound way. We should be able to rewrite the output as a sum of individual terms, I told myself.

  For the record, I did not assume MLP neurons are monosemantic or polysemantic, and this is why I did not mention SAEs.

Logit Prisms: De­com­pos­ing Trans­former Out­puts for Mechanis­tic Interpretability

Ex­plor­ing Llama-3-8B MLP Neurons