Neel Nanda comments on Refusal in LLMs is mediated by a single direction

[Neel Nanda]

As far as I’m aware, major open source chat tuned models like LLaMA are fine-tuned properly, not via a LoRA