I understand that these are working with public checkpoints but I’d be interested if you have internal models to see similar statistics for the size of weight updates, both across the training run, and within short periods, to see if there are correlations between which weights are updated. Do you get quite consistent, smooth updates, or can you find little clusters where connected weights all change substantially in just a few steps?
If there are moments of large updates it’d be interesting if you could look for what has changed (find sequences by maximising product of difference in likelihood between the two models and likelihood of the sequence as determined by final model?? anyway..)
Also I think the axes in the first graphs of ‘power law weight spectra..’ are mislabelled, should be rank/singular value?
I understand that these are working with public checkpoints but I’d be interested if you have internal models to see similar statistics for the size of weight updates, both across the training run, and within short periods, to see if there are correlations between which weights are updated. Do you get quite consistent, smooth updates, or can you find little clusters where connected weights all change substantially in just a few steps?
We do have internal models and we have run similar analyses on them. For obvious reasons I can’t say too much about this, but in general what we find is similar to the Pythia models. I think the effects I describe here are pretty general across quite a wide range of LLM architectures. Generally most changes are quite smooth it seems for both Pythia and other models. Haven’t looked much at correlations between specific weights so can’t say much about that.
Also I think the axes in the first graphs of ‘power law weight spectra..’ are mislabelled, should be rank/singular value?
Thanks for this! This is indeed the case. Am regenerating these plots and will update.
Commented on the last post but disappeared.
I understand that these are working with public checkpoints but I’d be interested if you have internal models to see similar statistics for the size of weight updates, both across the training run, and within short periods, to see if there are correlations between which weights are updated. Do you get quite consistent, smooth updates, or can you find little clusters where connected weights all change substantially in just a few steps?
If there are moments of large updates it’d be interesting if you could look for what has changed (find sequences by maximising product of difference in likelihood between the two models and likelihood of the sequence as determined by final model?? anyway..)
Also I think the axes in the first graphs of ‘power law weight spectra..’ are mislabelled, should be rank/singular value?
We do have internal models and we have run similar analyses on them. For obvious reasons I can’t say too much about this, but in general what we find is similar to the Pythia models. I think the effects I describe here are pretty general across quite a wide range of LLM architectures. Generally most changes are quite smooth it seems for both Pythia and other models. Haven’t looked much at correlations between specific weights so can’t say much about that.
Thanks for this! This is indeed the case. Am regenerating these plots and will update.