Vladimir makes an excellent point that it’s simply too soon to tell whether the next gen (eg gpt5) of LLMs will fizzle. I do think there’s reasonable evidence for suspecting that the generation AFTER that (eg gpt6) won’t be a straightforward scale up of gpt4.
I think we’re in a compute and data overhang for AGI, and that further parameter, compute, and data scaling beyond gpt5 level would be a waste of money.
The real question is whether gpt5 gen models will be just enough more capable than current ones to substantially increase the rate of the true limiting factor: algorithmic improvement.
https://www.lesswrong.com/posts/NRZfxAJztvx2ES5LG/a-path-to-human-autonomy
Vladimir makes an excellent point that it’s simply too soon to tell whether the next gen (eg gpt5) of LLMs will fizzle. I do think there’s reasonable evidence for suspecting that the generation AFTER that (eg gpt6) won’t be a straightforward scale up of gpt4. I think we’re in a compute and data overhang for AGI, and that further parameter, compute, and data scaling beyond gpt5 level would be a waste of money.
The real question is whether gpt5 gen models will be just enough more capable than current ones to substantially increase the rate of the true limiting factor: algorithmic improvement.