That does seem like the right threshold under any reasonable meaning of catastrophic, so long as it is understood that once found no patch can address the issue. The next level of models may or may not reach ASL-3. My guess is a 4.5-level model mostly wouldn’t count, a 5-level model mostly would count.
My current not-strongly-held guess is that something like ‘GPT-5’ won’t be ASL-3, but will be close and will be above a threshold where it will be significantly useful in producing the next generation. Something I’d be willing to call a weak early form of recursive self-improvement. Thus, I would expect the next generation, ‘GPT-6’, to be clearly into ASL-3 and able to do quite a bit of useful R&D work. Thus, I would expect a ‘GPT-7’ to be ASL-4. If each of these steps takes 1.5 years, that’s really not long until ASL-4.
My current not-strongly-held guess is that something like ‘GPT-5’ won’t be ASL-3, but will be close and will be above a threshold where it will be significantly useful in producing the next generation. Something I’d be willing to call a weak early form of recursive self-improvement. Thus, I would expect the next generation, ‘GPT-6’, to be clearly into ASL-3 and able to do quite a bit of useful R&D work. Thus, I would expect a ‘GPT-7’ to be ASL-4. If each of these steps takes 1.5 years, that’s really not long until ASL-4.
See my related prediction market here: https://manifold.markets/NathanHelmBurger/will-gpt5-be-capable-of-recursive-s?r=TmF0aGFuSGVsbUJ1cmdlcg
(market makes a slightly stronger case than stated here, in order to be controversial enough to be interesting)