Wouldn’t you expect (the many) current attempts to agentize LLMs to eat up a lot of the ‘agency overhang’? Especially since, something like the reflection/planning loops of agentized LLMs seem to me like a pretty plausible description of what human brains might be doing (e.g. system 2 / system 1, or see many of Seth Herd’s recent writings on agentized / scaffolded LLMs and similarities to cognitive architectures).
I don’t think the current attempts have eaten the agency overhang at all. Basically none of them have worked, so the agency advantage hasn’t been realized. But the public efforts just haven’t put that much person-power into improving memory or executive function systems.
So I’m predicting a discontinuity in capabilities just like Thane is suggesting. I wrote another short post trying to capture the cognitive intuition: Sapience, understanding, and “AGI” I think it might be a bit less sharp, since you might get an agent sort-of-working before it works really well. But the agency overhang is still there right now.
Wouldn’t you expect (the many) current attempts to agentize LLMs to eat up a lot of the ‘agency overhang’? Especially since, something like the reflection/planning loops of agentized LLMs seem to me like a pretty plausible description of what human brains might be doing (e.g. system 2 / system 1, or see many of Seth Herd’s recent writings on agentized / scaffolded LLMs and similarities to cognitive architectures).
I don’t think the current attempts have eaten the agency overhang at all. Basically none of them have worked, so the agency advantage hasn’t been realized. But the public efforts just haven’t put that much person-power into improving memory or executive function systems.
So I’m predicting a discontinuity in capabilities just like Thane is suggesting. I wrote another short post trying to capture the cognitive intuition: Sapience, understanding, and “AGI” I think it might be a bit less sharp, since you might get an agent sort-of-working before it works really well. But the agency overhang is still there right now.