I agree with you that human level on text prediction is not the same as human level generally, and that we might get the former long before we get the latter. Nevertheless, I think it’s possible that the two will come together—or, more cautiously, that something human-level at task prediction would also be transformative. (Maybe the text predictor by itself wouldn’t be an agent, but the text predictor could be re-trained as an agent fairly easily, or combined into a larger system that uses tree search or something and thus is an agent. Maybe it would be good enough at enough things that it would make a ridiculous amount of money automating away many jobs and inspire a huge boost in investment, which would then lead to human-level AGI. Or maybe it would be really good at persuasion… I’m going to write a post about persuasion tools someday soon.)
I think that it could plausibly be quite transformative in a TAI sense and occur over the next ten years, so perhaps we don’t have all that much of a disagreement on that point. I also think (just because we don’t have an especially clear idea of how modular intelligence is) that it could be quite uniform and a text predictor could surprise us with humanlike planning.
Maybe the text predictor by itself wouldn’t be an agent, but the text predictor could be re-trained as an agent fairly easily, or combined into a larger system that uses tree search or something and thus is an agent.
This maybe reflects a difference in intuition about how difficult agentive behaviour is to reach rather than language understanding. I would expect a simple tree search algorithm powered by GPT-6 to be… a model with humanlike language comprehension and incredibly dumb agentive behaviour, and that it wouldn’t be able to leverage the ‘intelligence’ of the language model in any significant way, because I see that as a seperate problem requiring seperate, difficult work. But I could be wrong.
I think there is a potential bias in that human-like language understanding and agentive behaviour have always gone together in human beings—we have no idea what a human-level language model that wasn’t human-level intelligent would be like. Since we can’t imagine it, we tend to default to imagining a human-in-a-box. I’m trying to correct for this bias by imagining that it might be quite different.
Nice, that’s probably a crux for us. I would expect tree search powered by GPT-6 to be probably pretty agentic. Isn’t that how AlphaZero works? Tree search + a win probability predictor?
It may well be a crux—an efficient ‘tree search’ or a similar goal-directed wrapper around a GPT-based system, that can play a role in real-world open-ended planning (presumably planning for an agent to be effecting outcomes in the real world via its text generation), would have to cover continuous action spaces and possible states containing unknown and shifting sets of possible actions (unlike the discrete and small, relative to the real universe, action space of Go which is perfect for a tree search), running (or approximating running) millions of primitive steps (individual text generations and exchanges) into the future (for long-term planning towards e.g. a multi-decade goal like humans are capable of).
That sounds like a problem that’s at least as hard as a language-model ‘success probability predictor’ GPT-N (probably with reward-modelling help, so it can optimize for a specific goal with its text generation). Though such a system would still be highly transformative, if it was human-level at prediction.
To clarify, this is Transformative not ‘Radically Transformative’ - transformative like Nuclear Power/Weapons, not like a new Industrial Revolution or an intelligence explosion.
I would expect tree search powered by GPT-6 to be probably pretty agentic.
I could imagine (if you found a domain with a fairly constrained set of actions and states, but involved text prediction somehow) that you could get agentic behaviour out of a tree search like the ones we currently have + GPT-N + an RL wrapper around the GPT-N. That might well be quite transformative—could imagine it being very good for persuasion, for example.
I agree with you that human level on text prediction is not the same as human level generally, and that we might get the former long before we get the latter. Nevertheless, I think it’s possible that the two will come together—or, more cautiously, that something human-level at task prediction would also be transformative. (Maybe the text predictor by itself wouldn’t be an agent, but the text predictor could be re-trained as an agent fairly easily, or combined into a larger system that uses tree search or something and thus is an agent. Maybe it would be good enough at enough things that it would make a ridiculous amount of money automating away many jobs and inspire a huge boost in investment, which would then lead to human-level AGI. Or maybe it would be really good at persuasion… I’m going to write a post about persuasion tools someday soon.)
I think that it could plausibly be quite transformative in a TAI sense and occur over the next ten years, so perhaps we don’t have all that much of a disagreement on that point. I also think (just because we don’t have an especially clear idea of how modular intelligence is) that it could be quite uniform and a text predictor could surprise us with humanlike planning.
This maybe reflects a difference in intuition about how difficult agentive behaviour is to reach rather than language understanding. I would expect a simple tree search algorithm powered by GPT-6 to be… a model with humanlike language comprehension and incredibly dumb agentive behaviour, and that it wouldn’t be able to leverage the ‘intelligence’ of the language model in any significant way, because I see that as a seperate problem requiring seperate, difficult work. But I could be wrong.
I think there is a potential bias in that human-like language understanding and agentive behaviour have always gone together in human beings—we have no idea what a human-level language model that wasn’t human-level intelligent would be like. Since we can’t imagine it, we tend to default to imagining a human-in-a-box. I’m trying to correct for this bias by imagining that it might be quite different.
Nice, that’s probably a crux for us. I would expect tree search powered by GPT-6 to be probably pretty agentic. Isn’t that how AlphaZero works? Tree search + a win probability predictor?
It may well be a crux—an efficient ‘tree search’ or a similar goal-directed wrapper around a GPT-based system, that can play a role in real-world open-ended planning (presumably planning for an agent to be effecting outcomes in the real world via its text generation), would have to cover continuous action spaces and possible states containing unknown and shifting sets of possible actions (unlike the discrete and small, relative to the real universe, action space of Go which is perfect for a tree search), running (or approximating running) millions of primitive steps (individual text generations and exchanges) into the future (for long-term planning towards e.g. a multi-decade goal like humans are capable of).
That sounds like a problem that’s at least as hard as a language-model ‘success probability predictor’ GPT-N (probably with reward-modelling help, so it can optimize for a specific goal with its text generation). Though such a system would still be highly transformative, if it was human-level at prediction.
To clarify, this is Transformative not ‘Radically Transformative’ - transformative like Nuclear Power/Weapons, not like a new Industrial Revolution or an intelligence explosion.
I could imagine (if you found a domain with a fairly constrained set of actions and states, but involved text prediction somehow) that you could get agentic behaviour out of a tree search like the ones we currently have + GPT-N + an RL wrapper around the GPT-N. That might well be quite transformative—could imagine it being very good for persuasion, for example.