tdooms comments on Tokenized SAEs: Infusing per-token biases.

[tdooms]

This is a completely fair suggestion. I’ll look into training a fully-fledged SAE with the same number of features for the full training duration.