Now, when a QNI comes along, it doesn’t necessarily look like a discontinuity, because there might be a lot of work to bridge the distance between idea and implementation. And, this work involves a lot of small details. Because of this, the first version is probably often only a slight improvement on SOTA. So, I’m guessing that QNIs produce something more like a discontinuity in the derivative than a discontinuity in the SOTA itself.
Don’t have a great source for this at hand, but my impression is that seemingly-QNIs surprisingly often just power existing exponential trends, meaning no change in derivative (on a log graph).
(A random comment in support of this — I remember chip design expert Jim Keller saying on Lex Fridman’s podcast that Moore’s Law is just a bunch of separate s-curves, as they have to come up with new ideas to work through challenges to shrinking transistors, and the new techniques work for a range of scales and then have to be replaced with new new ideas.)
Not sure if this question is easily settled, but it might be a crux for various views — how often do QNIs actually change the slope of the curve?
Don’t have a great source for this at hand, but my impression is that seemingly-QNIs surprisingly often just power existing exponential trends, meaning no change in derivative (on a log graph).
(A random comment in support of this — I remember chip design expert Jim Keller saying on Lex Fridman’s podcast that Moore’s Law is just a bunch of separate s-curves, as they have to come up with new ideas to work through challenges to shrinking transistors, and the new techniques work for a range of scales and then have to be replaced with new new ideas.)
Not sure if this question is easily settled, but it might be a crux for various views — how often do QNIs actually change the slope of the curve?