Those I know who train large models seem to be very confident we will get 100 Trillion parameter models before the end of the decade, but do not seem to think it will happen, say, in the next 2 years.
FWIW if the current trend continues we will first see 1e14 parameter models in 2 to 4 years from now.
FWIW if the current trend continues we will first see 1e14 parameter models in 2 to 4 years from now.