Thanks! We did try to use it in the repeat setting to make the model produce more than a single token, but it did not work well.
And as far as I remember it also did not improve the meaning prompt much.
Dmitrii Kharlapenko
Karma: 73
Thanks! We did try to use it in the repeat setting to make the model produce more than a single token, but it did not work well.
And as far as I remember it also did not improve the meaning prompt much.
Do you mean SAE encoder weights by input features? We did not look into them.