Do you expect similar results (besides the fact that it would take longer to train / cost more) without using LoRA?
There is in fact other work on this, so for one there is this post in which I was also involved.There was also the recent release by Yang et al. They are using normal fine-tuning on a very small dataset https://arxiv.org/pdf/2310.02949.pdfSo yes, this works with normal fine-tuning as well
Do you expect similar results (besides the fact that it would take longer to train / cost more) without using LoRA?
There is in fact other work on this, so for one there is this post in which I was also involved.
There was also the recent release by Yang et al. They are using normal fine-tuning on a very small dataset https://arxiv.org/pdf/2310.02949.pdf
So yes, this works with normal fine-tuning as well