keeping in mind I haven’t gotten a chance to read the paper itself…
the learning process is the main breakthrough, because it creates agents that can generalise to multiple problems.
There are admittedly commonalities between the different problems (e.g. the physics), but the same learning process applied to something like board game data might make a “general board game player”, or perhaps even something like a “general logistical-pipeline-optimiser” on the right economic/business datasets.
The ability to train an agent to succeed on such diverse problems is a noticeable step towards AGI, though there’s little way of knowing how much it helps until we figure out the Solution itself.
keeping in mind I’ve only recently started studying ML and modern AI techniques…
Judging from what I’ve read here on LW, it’s maybe around 3/4ths as significant as GPT-3? I might be wrong here, though.
Judging from what I’ve read here on LW, it’s maybe around 3/4ths as significant as GPT-3? I might be wrong here, though.
Disclaimer to the effect that I’m not very experienced here either and might be wrong too, but I’m not sure that’s the right comparison. It seems to me like GPT-2 (or GPT, but I don’t know anything about it) was a breakthrough in having one model that’s good at learning on new tasks with little data, and GPT-3 was a breakthrough in showing how far capabilities like that can extend with greater compute. This feels more like the former than the latter, but also sounds more significant than GPT-2 from a pure generalizing capability standpoint, so maybe slightly more significant than GPT-2?
I have little to no technical knowledge when it comes to AI, so I have two questions here.
1.) Are the agents or the learning process that was constructed the focus of the paper?
2.) How major of a breakthrough is this toward AGI?
keeping in mind I haven’t gotten a chance to read the paper itself… the learning process is the main breakthrough, because it creates agents that can generalise to multiple problems. There are admittedly commonalities between the different problems (e.g. the physics), but the same learning process applied to something like board game data might make a “general board game player”, or perhaps even something like a “general logistical-pipeline-optimiser” on the right economic/business datasets. The ability to train an agent to succeed on such diverse problems is a noticeable step towards AGI, though there’s little way of knowing how much it helps until we figure out the Solution itself.
keeping in mind I’ve only recently started studying ML and modern AI techniques… Judging from what I’ve read here on LW, it’s maybe around 3/4ths as significant as GPT-3? I might be wrong here, though.
Disclaimer to the effect that I’m not very experienced here either and might be wrong too, but I’m not sure that’s the right comparison. It seems to me like GPT-2 (or GPT, but I don’t know anything about it) was a breakthrough in having one model that’s good at learning on new tasks with little data, and GPT-3 was a breakthrough in showing how far capabilities like that can extend with greater compute. This feels more like the former than the latter, but also sounds more significant than GPT-2 from a pure generalizing capability standpoint, so maybe slightly more significant than GPT-2?