Jett Janiak
Jett(Jett Mayzner)
Karma: 89
For the two sets of mess3 parameters I checked the stationary distribution was uniform.
AISC project: TinyEvals
Polysemantic Attention Head in a 4-Layer Transformer
The activation patching, causal tracing and resample ablation terms seem to be out of date, compared to how you define them in your post on attribution patching.
This is such a cool result! I tried to reproduce it in this notebook