I think until the last few years the common idea was that AGI would be something developed in the metaphorical basement and lead to a singularity in short order, similar to Eliezer’s concept of Seed AI. Maybe stuff like AlphaGo was interesting/alarming to us but it seemed to mostly be overlooked by the public and especially government.
It wasn’t really clear until ChatGPT that general-ish AI was going to be relevant to the public and government/regulators well before full AGI.
I personally was mostly ignoring Transformers until GPT-3 came along. (That was mostly because I believed that “true AI” had to be a recurrent machine, whereas Transformers were feed-forward, and I did not understand that Transformers emulated RNNs when used in the autoregressive mode (the paper explaining that was published shortly after GPT-3).) So I thought, Transformers were remarkable, but the “true route to AGI” was elsewhere.
Then GPT-3 achieved two “Holy Grails” widely believed to be “impossible”/”long in the future”. Namely, few shot learning (so like a human it could learn a pattern from a single exposure without long training) and semi-competent program synthesis (which was considered to be next to impossible because of fragility of computer code to noise such as one-letter changes, with this supposed near-impossibility of program synthesis being the key obstacle to recursive self-improvement and foom).
These two breakthroughs were the key reason why I updated, rather than general linguistic competence (which was indeed quite impressive in 2019 models already).
ChatGPT was released on November 30 2022, so it’s only been around 7 months. The older ones were GPT-2 and GPT-3 which got attention among AI-followers but were relatively unknown to the public—and again, it wasn’t obvious then when or if ordinary people would come to know or care about these advances.
I think until the last few years the common idea was that AGI would be something developed in the metaphorical basement and lead to a singularity in short order, similar to Eliezer’s concept of Seed AI. Maybe stuff like AlphaGo was interesting/alarming to us but it seemed to mostly be overlooked by the public and especially government.
It wasn’t really clear until ChatGPT that general-ish AI was going to be relevant to the public and government/regulators well before full AGI.
Someone else said similar about the basement possibility, which I did not know.
Interesting questions raised though: Even if it wasn’t clear until GPT, wouldn’t that still have left something like 2-3 years?
Granted that is not 10-20 years.
It seems we all, collectively, did not update nearly enough on ChatGPT-2?
I personally was mostly ignoring Transformers until GPT-3 came along. (That was mostly because I believed that “true AI” had to be a recurrent machine, whereas Transformers were feed-forward, and I did not understand that Transformers emulated RNNs when used in the autoregressive mode (the paper explaining that was published shortly after GPT-3).) So I thought, Transformers were remarkable, but the “true route to AGI” was elsewhere.
Then GPT-3 achieved two “Holy Grails” widely believed to be “impossible”/”long in the future”. Namely, few shot learning (so like a human it could learn a pattern from a single exposure without long training) and semi-competent program synthesis (which was considered to be next to impossible because of fragility of computer code to noise such as one-letter changes, with this supposed near-impossibility of program synthesis being the key obstacle to recursive self-improvement and foom).
These two breakthroughs were the key reason why I updated, rather than general linguistic competence (which was indeed quite impressive in 2019 models already).
ChatGPT was released on November 30 2022, so it’s only been around 7 months. The older ones were GPT-2 and GPT-3 which got attention among AI-followers but were relatively unknown to the public—and again, it wasn’t obvious then when or if ordinary people would come to know or care about these advances.