Fine-tuning, whether using RL or not, is the proverbial “cherry on the cake” and the pre-trained model captures more than 99.9% of the intelligence of the model.
I am still amazed by the strength of general models. There is the no-free lunch theorem that people use to point out that we will probably have specialized AI’s because they will be better. Current practice seems to contradict this.
I have yet to see an interesting implication of the “no free lunch” theorem. But the world we move to seems to be of general foundation models that can be combined with a variety of tailor-made adapters (e.g. LORA weights or prompts) that help them tackle any particular application. The general model is the “operating system” and the adapters are the “apps”.
This emphasis on generality makes deployment of future models a lot easier. We first build a gpt4 ecosystem. When gpt5 comes out it will be easy to implement (e.g. autogpt can run just as easy on gpt4 as on gpt5). The adaptions that are necessary are very small and thus very fast deployment of future models is to be expected.
Yes. Right now we would have to re-train all LORA weights of a model when an updated version comes out, but I imagine that at some point we would have “transpilers” for adaptors that don’t use natural language as their API as well.
I am still amazed by the strength of general models. There is the no-free lunch theorem that people use to point out that we will probably have specialized AI’s because they will be better. Current practice seems to contradict this.
I have yet to see an interesting implication of the “no free lunch” theorem. But the world we move to seems to be of general foundation models that can be combined with a variety of tailor-made adapters (e.g. LORA weights or prompts) that help them tackle any particular application. The general model is the “operating system” and the adapters are the “apps”.
This emphasis on generality makes deployment of future models a lot easier. We first build a gpt4 ecosystem. When gpt5 comes out it will be easy to implement (e.g. autogpt can run just as easy on gpt4 as on gpt5). The adaptions that are necessary are very small and thus very fast deployment of future models is to be expected.
Yes. Right now we would have to re-train all LORA weights of a model when an updated version comes out, but I imagine that at some point we would have “transpilers” for adaptors that don’t use natural language as their API as well.