I disagree with that view, primarily due to my belief that the sharp left turn is an old remnant of the hard-takeoff view that was always physically problematic, and now that we actually have AI in a limited form, while there does seem to be a discontinuity at first, rarely will it get you all the way, and once we lose that first instance, progress is much more smooth and slow. So slow-takeoff is my mainline scenario for AI.
Finally, I think we will ultimately have to experiment, because being very blunt, humans are quite bad at reasoning from first principles or priors, and without feedback from reality, reasoning like a formal mathematician or first principles tends to be wildly wrong for real life.
I disagree with that view, primarily due to my belief that the sharp left turn is an old remnant of the hard-takeoff view that was always physically problematic, and now that we actually have AI in a limited form, while there does seem to be a discontinuity at first, rarely will it get you all the way, and once we lose that first instance, progress is much more smooth and slow. So slow-takeoff is my mainline scenario for AI.
Finally, I think we will ultimately have to experiment, because being very blunt, humans are quite bad at reasoning from first principles or priors, and without feedback from reality, reasoning like a formal mathematician or first principles tends to be wildly wrong for real life.