Okay, so you know how AI today isn’t great at certain… let’s say “long-horizon” tasks? Like novel large-scale engineering projects, or writing a long book series with lots of foreshadowing?
(Modulo the fact that it can play chess pretty well, which is longer-horizon than some things; this distinction is quantitative rather than qualitative and it’s being eroded, etc.)
And you know how the AI doesn’t seem to have all that much “want”- or “desire”-like behavior?
(Modulo, e.g., the fact that it can play chess pretty well, which indicates a certain type of want-like behavior in the behaviorist sense. An AI’s ability to win no matter how you move is the same as its ability to reliably steer the game-board into states where you’re check-mated, as though it had an internal check-mating “goal” it were trying to achieve. This is again a quantitative gap that’s being eroded.)
I don’t think the following is all that relevant to the point you are making in this post, but someone cited this post of yours in relation to the question of whether LLMs are “intelligent” (summarizing the post as “Nate says LLMs aren’t intelligent”) and then argued against the post as goalpost-moving, so I wanted to discuss that.
It may come as a shock to some, that Abram Demski adamantly defends the following position: GPT4 is AGI. I would be goalpost-moving if I said otherwise. I think the AGI community is goalpost-moving to the extent that it says otherwise.
I think there is some tendency in the AI Risk community to equate “AGI” with “the sort of AI which kills all the humans unless it is aligned”. But “AGI” stands for “artificial general intelligence”, not “kills all the humans”. I think it makes more sense for the definition of AGI to be up to the community of AI researchers who use the term AGI to distance their work from narrow AI, rather than for it to be up to the AI risk community. And GPT4 is definitely not narrow AI.
I’ll argue an even stronger claim: if you come up with a task which can be described and completed entirely in text format (and then evaluated somehow for performance quality), for most such tasks the performance of GPT4 is at or above the performance of a random human. (We can even be nice and only randomly sample humans who speak whichever languages are appropriate to the task; I’ll still stand by the claim.) Yes, GPT4 has some weaknesses compared to a random human. But most claims of weaknesses I’ve heard are in fact contrasting GPT4 to expert humans, not random humans. So my stronger claim is: GPT4 is human-level AGI, maybe not by all possible definitions of the term, but by a very reasonable-seeming definition which 2014 Abram Demski might have been perfectly happy with. To deny this would be goalpost-moving for me; and, I expect, for many.
So (and I don’t think this is what you were saying) if GPT4 were being ruled out of “human-level AGI” because it cannot write a coherent set of novels on its own, or do a big engineering project, well, I call shenanigans. Most humans can’t do that either.
GPT-4 as a human level AGI is reasonable as a matter of evaluating the meaning of words, but this meaning of “AGI” doesn’t cut reality at its joints. Humans are a big deal not for the reason of being at human level, but because there is capability for unbounded technological progress, including through building superintelligence. Ability for such progress doesn’t require being superintelligent, so it’s a different thing. For purposes of AI timelines it’s the point where history starts progressing at AI speed rather than at human speed. There should be a name for this natural concept, and “AGI” seems like a reasonable option.
I don’t think the following is all that relevant to the point you are making in this post, but someone cited this post of yours in relation to the question of whether LLMs are “intelligent” (summarizing the post as “Nate says LLMs aren’t intelligent”) and then argued against the post as goalpost-moving, so I wanted to discuss that.
It may come as a shock to some, that Abram Demski adamantly defends the following position: GPT4 is AGI. I would be goalpost-moving if I said otherwise. I think the AGI community is goalpost-moving to the extent that it says otherwise.
I think there is some tendency in the AI Risk community to equate “AGI” with “the sort of AI which kills all the humans unless it is aligned”. But “AGI” stands for “artificial general intelligence”, not “kills all the humans”. I think it makes more sense for the definition of AGI to be up to the community of AI researchers who use the term AGI to distance their work from narrow AI, rather than for it to be up to the AI risk community. And GPT4 is definitely not narrow AI.
I’ll argue an even stronger claim: if you come up with a task which can be described and completed entirely in text format (and then evaluated somehow for performance quality), for most such tasks the performance of GPT4 is at or above the performance of a random human. (We can even be nice and only randomly sample humans who speak whichever languages are appropriate to the task; I’ll still stand by the claim.) Yes, GPT4 has some weaknesses compared to a random human. But most claims of weaknesses I’ve heard are in fact contrasting GPT4 to expert humans, not random humans. So my stronger claim is: GPT4 is human-level AGI, maybe not by all possible definitions of the term, but by a very reasonable-seeming definition which 2014 Abram Demski might have been perfectly happy with. To deny this would be goalpost-moving for me; and, I expect, for many.
So (and I don’t think this is what you were saying) if GPT4 were being ruled out of “human-level AGI” because it cannot write a coherent set of novels on its own, or do a big engineering project, well, I call shenanigans. Most humans can’t do that either.
GPT-4 as a human level AGI is reasonable as a matter of evaluating the meaning of words, but this meaning of “AGI” doesn’t cut reality at its joints. Humans are a big deal not for the reason of being at human level, but because there is capability for unbounded technological progress, including through building superintelligence. Ability for such progress doesn’t require being superintelligent, so it’s a different thing. For purposes of AI timelines it’s the point where history starts progressing at AI speed rather than at human speed. There should be a name for this natural concept, and “AGI” seems like a reasonable option.
I agree that this is an important distinction, but I personally prefer to call it “transformative AI” or some such.