I admit, I stopped reading the linked paper when I saw the page count, but I don’t see why you’re rejecting decades of 60<IQ<100 AIs as implausible (uninteresting is another matter, but some people are interested). An IQ70 AI is little more able to self-improve than a IQ70 human is able to improve an AI. Even an IQ120 human would have trouble with that. The task of bringing AIs from IQ60 to IQ140 where they can start meaningfully contributing to AI research falls to IQ180 humans, and will probably take a long time.
Not that talking about the IQ of an AI makes a whole lot of sense—mindspace is many-dimensional and no AI is likely to land on the human manifold. But as a very, very crude approximation, it will do here.
An IQ70 AI is little more able to self-improve than a IQ70 human is able to improve an AI
This is not obviously true. We’re a lot less well optimized for improving code than some conceivable AIs can be: a seed AI with relatively modest general intelligence but very good self-modification heuristics might still end up knocking our socks off.
That said, there’s a much larger design space where this isn’t the case.
My [unverified] intuition on AI properties is that the delta between current status and ‘IQ60AI’ is multiple orders of magnitude larger than the delta between ‘IQ60AI’ and ‘IQ180AI’. In essence, there is not that much “mental horsepower” difference between the stereotypical Einstein and a below-average person; it doesn’t require a much larger brain or completely different neuronal wiring or a million years of evolutionary tuning.
We don’t know how to get to IQ60AI; but getting from IQ60AI to IQ180AI could (IMHO) be done with currently known methods in many labs around the world by the current (non IQ180) researchers rapidly (ballpark of 6 months maybe?). We know from history that a 0 IQ process can optimize from monkey-level intelligence to an Einstein by bruteforcing; So in essence, if you’ve got IQ70 minds that can be rapidly run and simulted, then just apply more hardware (for more time-compression) and optimization, as that gap seems to require exactly 0 significant breakthroughs to get to IQ180.
I admit, I stopped reading the linked paper when I saw the page count, but I don’t see why you’re rejecting decades of 60<IQ<100 AIs as implausible (uninteresting is another matter, but some people are interested). An IQ70 AI is little more able to self-improve than a IQ70 human is able to improve an AI. Even an IQ120 human would have trouble with that. The task of bringing AIs from IQ60 to IQ140 where they can start meaningfully contributing to AI research falls to IQ180 humans, and will probably take a long time.
Not that talking about the IQ of an AI makes a whole lot of sense—mindspace is many-dimensional and no AI is likely to land on the human manifold. But as a very, very crude approximation, it will do here.
This is not obviously true. We’re a lot less well optimized for improving code than some conceivable AIs can be: a seed AI with relatively modest general intelligence but very good self-modification heuristics might still end up knocking our socks off.
That said, there’s a much larger design space where this isn’t the case.
Human programmers at IQ 70 also can’t be run anywhere near as fast as AI programmers at IQ 70.
My [unverified] intuition on AI properties is that the delta between current status and ‘IQ60AI’ is multiple orders of magnitude larger than the delta between ‘IQ60AI’ and ‘IQ180AI’. In essence, there is not that much “mental horsepower” difference between the stereotypical Einstein and a below-average person; it doesn’t require a much larger brain or completely different neuronal wiring or a million years of evolutionary tuning.
We don’t know how to get to IQ60AI; but getting from IQ60AI to IQ180AI could (IMHO) be done with currently known methods in many labs around the world by the current (non IQ180) researchers rapidly (ballpark of 6 months maybe?). We know from history that a 0 IQ process can optimize from monkey-level intelligence to an Einstein by bruteforcing; So in essence, if you’ve got IQ70 minds that can be rapidly run and simulted, then just apply more hardware (for more time-compression) and optimization, as that gap seems to require exactly 0 significant breakthroughs to get to IQ180.