Like if I want to experiment with a steering technique, it would be useful to have a language model that is small, capable, but not so finetuned that it becomes inflexible. (Or maybe ideally, a model which has both a finetuned and a non-finetuned variant.)
I’ve seen some people use GPT-2. Is that recommended? Are there any alternatives?
[Question] What are some good language models to experiment with?
Like if I want to experiment with a steering technique, it would be useful to have a language model that is small, capable, but not so finetuned that it becomes inflexible. (Or maybe ideally, a model which has both a finetuned and a non-finetuned variant.)
I’ve seen some people use GPT-2. Is that recommended? Are there any alternatives?