IMO, soft/smooth/gradual still convey wrong impressions. They still sound like “slow takeoff”, they sound like the progress would be steady enough that normal people would have time to orient to what’s happening, keep track, and exert control. As you’re pointing out, that’s not necessarily the case at all: from a normal person’s perspective, this scenario may very much look very sharp and abrupt.
The main difference in this classification seems to be whether AI progress occurs “externally”, as part of economic and R&D ecosystems, or “internally”, as part of an opaque self-improvement process within a (set of) AI system(s). (Though IMO there’s a mostly smooth continuum of scenarios, and I don’t know that there’s a meaningful distinction/clustering at all.)
From this perspective, even continuous vs. discontinuous don’t really cleave reality at the joints. The self-improvement is still “continuous” (or, more accurately, incremental) in the hard-takeoff/RSI case, from the AI’s own perspective. It’s just that ~nothing besides the AI itself is relevant to the process.
Just “external” vs. “internal” takeoff, maybe? “Economic” vs. “unilateral”?
I do agree with that, although I don’t know that I feel the need to micromanage the implicature of the term that much.
I think it’s good to try to find terms that don’t have misleading connotations, but also good not to fight too hard to control the exact political implications of a term, partly because there’s not a clear cutoff between being clear and being actively manipulative (and not obvious to other people which you’re being, esp. if they disagree with you about the implications), and partly because there’s a bit of a red queen race of trying to get terms into common parlance that benefit your agenda, and, like, let’s just not.
Fast/slow just felt actively misleading.
I think the terms you propose here are interesting but a bit too opinionated about the mechanism involved. I’m not that confident those particular mechanisms will turn out to be decisive, and don’t think the mechanism is actually that cruxy for what the term implies in terms of strategy.
If I did want to try to give it the connotations that actually feel right to me, I might say “rolling*” as the “smooth” option. I don’t have a great “fast” one.
*although someone just said they found “rolling” unintuitive so shrug.
IMO, soft/smooth/gradual still convey wrong impressions. They still sound like “slow takeoff”, they sound like the progress would be steady enough that normal people would have time to orient to what’s happening, keep track, and exert control. As you’re pointing out, that’s not necessarily the case at all: from a normal person’s perspective, this scenario may very much look very sharp and abrupt.
The main difference in this classification seems to be whether AI progress occurs “externally”, as part of economic and R&D ecosystems, or “internally”, as part of an opaque self-improvement process within a (set of) AI system(s). (Though IMO there’s a mostly smooth continuum of scenarios, and I don’t know that there’s a meaningful distinction/clustering at all.)
From this perspective, even continuous vs. discontinuous don’t really cleave reality at the joints. The self-improvement is still “continuous” (or, more accurately, incremental) in the hard-takeoff/RSI case, from the AI’s own perspective. It’s just that ~nothing besides the AI itself is relevant to the process.
Just “external” vs. “internal” takeoff, maybe? “Economic” vs. “unilateral”?
I do agree with that, although I don’t know that I feel the need to micromanage the implicature of the term that much.
I think it’s good to try to find terms that don’t have misleading connotations, but also good not to fight too hard to control the exact political implications of a term, partly because there’s not a clear cutoff between being clear and being actively manipulative (and not obvious to other people which you’re being, esp. if they disagree with you about the implications), and partly because there’s a bit of a red queen race of trying to get terms into common parlance that benefit your agenda, and, like, let’s just not.
Fast/slow just felt actively misleading.
I think the terms you propose here are interesting but a bit too opinionated about the mechanism involved. I’m not that confident those particular mechanisms will turn out to be decisive, and don’t think the mechanism is actually that cruxy for what the term implies in terms of strategy.
If I did want to try to give it the connotations that actually feel right to me, I might say “rolling*” as the “smooth” option. I don’t have a great “fast” one.
*although someone just said they found “rolling” unintuitive so shrug.
Assuming this is the important distinction, I like something like “isolated”/“integrated” better than either of those.