Although the training process, in theory, can be wholly defined by source code, this is generally not practical, because doing so would require releasing (1) the methods used to train the model, (2) all data used to train the model, and (3) so called “training checkpoints” which are snapshots of the state of the model at various points in the training process.
Exactly. Without the data, the model design cannot be trained again, and you end up fine-tuning a black box (the “open weights”).
Exactly. Without the data, the model design cannot be trained again, and you end up fine-tuning a black box (the “open weights”).
Thanks for writing this.