Simon Lermen comments on LoRA Fine-tuning Efficiently Undoes Safety Training from Llama 2-Chat 70B

[Simon Lermen]

There is a paper out on the exact phenomenon you noticed:

https://​​arxiv.org/​​abs/​​2310.03693