I don’t agree that there is no conceivable path forward with current technology. This perspective seems too focused on base LLM models diminishing returns (eg 4.5 to 4). You brought up CoT and limited reasoning window, but I could imagine this solved pretty easily with some type of master / sub task layering. I also believe some of those issues could in fact be solved with brute scale anyway. You also critique the newer models as “Frankenstein” but I think OAI is right about that as an evolution. Basic models should have basic inputs and output functionality like we have for computers. “Models” don’t need to be pure token generators and can have templates: that is good and fine. A model should eg “clean” or interpret user inputs before acting on them and that can be two separate functions. Also, some of the diminishing returns you notice in text output are a consequence of the mode perfecting; there are no grammar or text mistakes, writing is rich and structured, and it even has markdown.
When we talk about AGI (not ASI) this is a line I think we have already effectively crossed. We don’t have sentient AI, but we have human level reasoning and intelligence that can act as directed on 99% of human domain tasks. Whether or not the AI questions life on its own isn’t qualifier.
For ASI, here are my thoughts: o series models are effective step functions (or patterns of subprocesses or templates). These templates are human defined reasoning currently, as someone at OAI probably wrote them. If an AI orchestrator can begin experimenting on and updating its own logic functions iteratively, we might see novel thought patterns form, sub functions we don’t fully understand working in concert, and the forming of new reasoning patterns on its own. There is still some theoretical boundary where this process begins self-replicating / adapting on its own, which may still be ways away, but at that point it is a matter of feeding it energy and compute resources and letting it do its thing. Realistically, I doubt we will ever require a scale of energy so unreasonable as to stop progress, assuming we see Moores law improvements like we do with everything else in tech.
Ultimately I think you’re not valuing how effective functions working together in concert could be in overcoming the few, finer frictions that remain, and counter that I could conceive of a realistic pathway to ASI without needing that many further breakthroughs
I don’t agree that there is no conceivable path forward with current technology. This perspective seems too focused on base LLM models diminishing returns (eg 4.5 to 4). You brought up CoT and limited reasoning window, but I could imagine this solved pretty easily with some type of master / sub task layering. I also believe some of those issues could in fact be solved with brute scale anyway. You also critique the newer models as “Frankenstein” but I think OAI is right about that as an evolution. Basic models should have basic inputs and output functionality like we have for computers. “Models” don’t need to be pure token generators and can have templates: that is good and fine. A model should eg “clean” or interpret user inputs before acting on them and that can be two separate functions. Also, some of the diminishing returns you notice in text output are a consequence of the mode perfecting; there are no grammar or text mistakes, writing is rich and structured, and it even has markdown.
When we talk about AGI (not ASI) this is a line I think we have already effectively crossed. We don’t have sentient AI, but we have human level reasoning and intelligence that can act as directed on 99% of human domain tasks. Whether or not the AI questions life on its own isn’t qualifier.
For ASI, here are my thoughts: o series models are effective step functions (or patterns of subprocesses or templates). These templates are human defined reasoning currently, as someone at OAI probably wrote them. If an AI orchestrator can begin experimenting on and updating its own logic functions iteratively, we might see novel thought patterns form, sub functions we don’t fully understand working in concert, and the forming of new reasoning patterns on its own. There is still some theoretical boundary where this process begins self-replicating / adapting on its own, which may still be ways away, but at that point it is a matter of feeding it energy and compute resources and letting it do its thing. Realistically, I doubt we will ever require a scale of energy so unreasonable as to stop progress, assuming we see Moores law improvements like we do with everything else in tech.
Ultimately I think you’re not valuing how effective functions working together in concert could be in overcoming the few, finer frictions that remain, and counter that I could conceive of a realistic pathway to ASI without needing that many further breakthroughs