Existing large tech companies are using approaches like this, training or fine-tuning small models on data generated by large ones.
For example, it’s helpful for the cold start problem, where you don’t yet have user input to train/fine-tune your small model on because the product the model is intended for hasn’t been launched yet: have a large model create some simulated user input, train the small model on that, launch a beta test, and then retrain your small model with real user input as soon as you have some.
Existing large tech companies are using approaches like this, training or fine-tuning small models on data generated by large ones.
For example, it’s helpful for the cold start problem, where you don’t yet have user input to train/fine-tune your small model on because the product the model is intended for hasn’t been launched yet: have a large model create some simulated user input, train the small model on that, launch a beta test, and then retrain your small model with real user input as soon as you have some.