Ben Adlam (via Maloney et al 2022) makes an interesting point: if you plot parameters vs training data, it’s a nearly perfect 1:1 ratio historically. (He doesn’t seem to have published anything formally on this.)
We have conveniently just updated our database if anyone wants to investigate this further!https://epochai.org/data/notable-ai-models
Ben Adlam (via Maloney et al 2022) makes an interesting point: if you plot parameters vs training data, it’s a nearly perfect 1:1 ratio historically. (He doesn’t seem to have published anything formally on this.)
We have conveniently just updated our database if anyone wants to investigate this further!
https://epochai.org/data/notable-ai-models