It’s easy to lose sight of the common threads in these arguments and that the commonalities largely outweigh the disagreements. Your comparisons with the different technologies (guns, nukes) was especially useful and I hadn’t seen this framed explicitly, with the low effort vs fundamental technology aspect. One thought I had was that this potentially could play out somewhere in the middle; where there is continuous progress with increasingly powerful and disruptive AI to a point, but with the arrival of self-awareness/self-modification—since a self-adaptive technology could be considered itself a fundamental technology—something of a phase transition could force a minimal discontinuity in rate of change. So perhaps continuous with occasional peaks.
Although in practice this would likely just appear as a continuity and not really make much of a difference.
Excellent post. Very useful.
It’s easy to lose sight of the common threads
in these arguments and that the commonalities largely outweigh the disagreements. Your comparisons with the different technologies (guns, nukes) was especially useful and I hadn’t seen this framed explicitly, with the low effort vs fundamental technology aspect. One thought I had was that this potentially could play out somewhere in the middle; where there is continuous progress with increasingly powerful and disruptive AI to a point, but with the arrival of self-awareness/self-modification—since a self-adaptive technology could be considered itself a fundamental technology—something of a phase transition could force a minimal discontinuity in rate of change. So perhaps continuous with occasional peaks.Although in practice this would likely just appear as a continuity and not really make much of a difference.