These sorts of models all seem to be heavily dependent on “borrowing” a ton of intelligence from humans. To me they don’t seem likely to be capable of gaining any new skills that humans don’t already possess and give lots of demonstrations of. As such they don’t really seem to be FOOMy to me.
Also they’re literally reliant on human language descriptions of what they’re gonna do and why they’re gonna do it.
To clarify I think this is clearer evidence of ~AGI than FOOM. If anything, AGI seeming this straightforward might be an update against “ASI follows quickly from AGI”, but then again it’s not clear.
Insofar as totally new skills in a domain without data is basically as hard as RL, RL still doesn’t really work? Could this path towards AGI working be seen to provide evidence that RL might just be computationally as hard as brute force search, and that the existence of human intelligence shouldn’t make us think there’s some secret sauce we haven’t found?
(TBC you can still cause civilization-ending damage with large clusters of Von Neumann-esque AGIs, they don’t have to FOOM quickly.)
These sorts of models all seem to be heavily dependent on “borrowing” a ton of intelligence from humans. To me they don’t seem likely to be capable of gaining any new skills that humans don’t already possess and give lots of demonstrations of. As such they don’t really seem to be FOOMy to me.
Also they’re literally reliant on human language descriptions of what they’re gonna do and why they’re gonna do it.
Reflecting on this and other comments, I decided to edit the original post to retract the call for a “fire alarm”.
To clarify I think this is clearer evidence of ~AGI than FOOM. If anything, AGI seeming this straightforward might be an update against “ASI follows quickly from AGI”, but then again it’s not clear.
Insofar as totally new skills in a domain without data is basically as hard as RL, RL still doesn’t really work? Could this path towards AGI working be seen to provide evidence that RL might just be computationally as hard as brute force search, and that the existence of human intelligence shouldn’t make us think there’s some secret sauce we haven’t found?
(TBC you can still cause civilization-ending damage with large clusters of Von Neumann-esque AGIs, they don’t have to FOOM quickly.)