“Current architecture” is a narrower category than “human-designed architecture.” You might have said in 2012 “current architectures can’t beat Go” but that wouldn’t have meant we needed RSI to beat Go[1]; we just need to design something better than what we had then.
I think it is likely that a human-designed architecture could be an extinction-level AI. I think it is not obvious whether the first extinction-level AI will be human-designed or AI-designed, as it’s both determined by technological uncertainties and political uncertainties.
I think it is plausible that if you did massive scaling on current architecture, you could get an extinction-level AI, but it is pretty unlikely that this will be the first or even seriously attempted. [Like, if we had a century of hardware progress and no software progress, could GPT-2123 be extinction-level? I’m not gonna rule it out, but I am going to consider extremely unlikely “a century of hardware progress with no software progress.”]
Does expert iteration count as recursive self-improvement? IMO “not really but it’s close.” Obviously it doesn’t let you overcome any architecture-imposed limitations, but it lets you iteratively improve based on your current capability level. And if you view some of our current training regimes as already RSI-like, then this conversation changes.
“Current architecture” is a narrower category than “human-designed architecture.” You might have said in 2012 “current architectures can’t beat Go” but that wouldn’t have meant we needed RSI to beat Go[1]; we just need to design something better than what we had then.
I think it is likely that a human-designed architecture could be an extinction-level AI. I think it is not obvious whether the first extinction-level AI will be human-designed or AI-designed, as it’s both determined by technological uncertainties and political uncertainties.
I think it is plausible that if you did massive scaling on current architecture, you could get an extinction-level AI, but it is pretty unlikely that this will be the first or even seriously attempted. [Like, if we had a century of hardware progress and no software progress, could GPT-2123 be extinction-level? I’m not gonna rule it out, but I am going to consider extremely unlikely “a century of hardware progress with no software progress.”]
Does expert iteration count as recursive self-improvement? IMO “not really but it’s close.” Obviously it doesn’t let you overcome any architecture-imposed limitations, but it lets you iteratively improve based on your current capability level. And if you view some of our current training regimes as already RSI-like, then this conversation changes.