Thanks for the excellent post! I don’t think you mentioned anything about technical improvements in training, such as efficiency of parallelization. Do you know if there are internet I f things going in there, or does that shake out as part of the GPU generational improvements?
Thanks for bringing this up. I don’t think I mentioned any algorithmic improvements apart from RETRO so these predictions are probably somewhat conservative.
Thanks for the excellent post! I don’t think you mentioned anything about technical improvements in training, such as efficiency of parallelization. Do you know if there are internet I f things going in there, or does that shake out as part of the GPU generational improvements?
Thanks for bringing this up. I don’t think I mentioned any algorithmic improvements apart from RETRO so these predictions are probably somewhat conservative.