In my video, I describe one of the breakthroughs in matrix multplication after Strassen as “Efficient parallelization, like MapReduce, in the nineties”. This insight is used in practice, though some of the other improvements I mention are not practical.
In the section “Finding the secret sauce”, you asked for a canonical historical example of an insight having immediate dramatic effects. The canonical example is “nuclear weapons”, but this does not seem to precisely satisfy your requirements. While this example is commonly used, I’m not too fond of it, which is why I substituted my own.
My video “If AGI was Matrix Multiplication” does not claim that that fast matrix multiplication is a particular impressive intellectual breakthrough. It is a moderate improvement, but I show that such moderate improvement are sufficient to trigger an intelligence explosion.
If we wish to predict the trajectory of improvements to the first AGI algorithm (hypothetically), we might choose as reference class “Trajectories of improvements to all problems”. With this reference class, it looks like most improvement happens slowly, continuously and with a greater emphasis on experience rather than insights.
We might instead choose the reference class “Trajectories of improvement to algorithms”, which is far narrower, but still rich in examples. Here a book on the history of algorithms will provide many examples of improvements due to difficult theory and clever insights, with matrix multiplication not standing out as particular impressive. Presumably, most of these trajectories are sufficient for an intelligence explosion, if the trajectory were to be followed by the first AGI algorithm. However, a history book is a highly biased view of the past, as it will tend to focus on the most impressive trajectories. I am unsure about how to overcome this problem.
An even narrower reference class would be “Trajectories of improvement to AI algorithms”, where training artificial neural networks is an example of a trajectory that would surely be explosive. I intuitively feel that this reference class is too narrow, as the first AGI algorithm could be substantially different from previous AI algorithms.
Wikipedia claims that “it is faster in cases where n > 100 or so” https://en.wikipedia.org/wiki/Matrix_multiplication_algorithm
The introduction of this Wikipedia article seems to describe these improvements as practically useful.
In my video, I describe one of the breakthroughs in matrix multplication after Strassen as “Efficient parallelization, like MapReduce, in the nineties”. This insight is used in practice, though some of the other improvements I mention are not practical.
In the section “Finding the secret sauce”, you asked for a canonical historical example of an insight having immediate dramatic effects. The canonical example is “nuclear weapons”, but this does not seem to precisely satisfy your requirements. While this example is commonly used, I’m not too fond of it, which is why I substituted my own.
My video “If AGI was Matrix Multiplication” does not claim that that fast matrix multiplication is a particular impressive intellectual breakthrough. It is a moderate improvement, but I show that such moderate improvement are sufficient to trigger an intelligence explosion.
If we wish to predict the trajectory of improvements to the first AGI algorithm (hypothetically), we might choose as reference class “Trajectories of improvements to all problems”. With this reference class, it looks like most improvement happens slowly, continuously and with a greater emphasis on experience rather than insights.
We might instead choose the reference class “Trajectories of improvement to algorithms”, which is far narrower, but still rich in examples. Here a book on the history of algorithms will provide many examples of improvements due to difficult theory and clever insights, with matrix multiplication not standing out as particular impressive. Presumably, most of these trajectories are sufficient for an intelligence explosion, if the trajectory were to be followed by the first AGI algorithm. However, a history book is a highly biased view of the past, as it will tend to focus on the most impressive trajectories. I am unsure about how to overcome this problem.
An even narrower reference class would be “Trajectories of improvement to AI algorithms”, where training artificial neural networks is an example of a trajectory that would surely be explosive. I intuitively feel that this reference class is too narrow, as the first AGI algorithm could be substantially different from previous AI algorithms.