IMO, soft/smooth/gradual still convey wrong impressions. They still sound like “slow takeoff”, they sound like the progress would be steady enough that normal people would have time to orient to what’s happening, keep track, and exert control. As you’re pointing out, that’s not necessarily the case at all: from a normal person’s perspective, this scenario may very much look very sharp and abrupt.
The main difference in this classification seems to be whether AI progress occurs “externally”, as part of economic and R&D ecosystems, or “internally”, as part of an opaque self-improvement process within a (set of) AI system(s). (Though IMO there’s a mostly smooth continuum of scenarios, and I don’t know that there’s a meaningful distinction/clustering at all.)
From this perspective, even continuous vs. discontinuous don’t really cleave reality at the joints. The self-improvement is still “continuous” (or, more accurately, incremental) in the hard-takeoff/RSI case, from the AI’s own perspective. It’s just that ~nothing besides the AI itself is relevant to the process.
Just “external” vs. “internal” takeoff, maybe? “Economic” vs. “unilateral”?
I do agree with that, although I don’t know that I feel the need to micromanage the implicature of the term that much.
I think it’s good to try to find terms that don’t have misleading connotations, but also good not to fight too hard to control the exact political implications of a term, partly because there’s not a clear cutoff between being clear and being actively manipulative (and not obvious to other people which you’re being, esp. if they disagree with you about the implications), and partly because there’s a bit of a red queen race of trying to get terms into common parlance that benefit your agenda, and, like, let’s just not.
Fast/slow just felt actively misleading.
I think the terms you propose here are interesting but a bit too opinionated about the mechanism involved. I’m not that confident those particular mechanisms will turn out to be decisive, and don’t think the mechanism is actually that cruxy for what the term implies in terms of strategy.
If I did want to try to give it the connotations that actually feel right to me, I might say “rolling*” as the “smooth” option. I don’t have a great “fast” one.
*although someone just said they found “rolling” unintuitive so shrug.
IMO, soft/smooth/gradual still convey wrong impressions. They still sound like “slow takeoff”, they sound like the progress would be steady enough that normal people would have time to orient to what’s happening, keep track, and exert control.
That is exactly the meaning that I’d thought was standard for “soft takeoff” (and which I assumed was synonymous with “slow takeoff”), e.g. as I wrote in 2012:
Bugaj and Goertzel (2007) consider three kinds of AGI scenarios: capped intelligence, soft takeoff, and hard takeoff. In a capped intelligence scenario, all AGIs are prevented from exceeding a predetermined level of intelligence and remain at a level
roughly comparable with humans. In a soft takeoff scenario, AGIs become far more
powerful than humans, but on a timescale which permits ongoing human interaction
during the ascent. Time is not of the essence, and learning proceeds at a relatively
human-like pace. In a hard takeoff scenario, an AGI will undergo an extraordinarily fast
increase in power, taking effective control of the world within a few years or less. [Footnote: Bugaj and Goertzel defined hard takeoff to refer to a period of months or less. We have chosen a
somewhat longer time period, as even a few years might easily turn out to be too little time for society to
properly react.] In
this scenario, there is little time for error correction or a gradual tuning of the AGI’s
goals.
(B&G didn’t actually invent soft/hard takeoff, but it was the most formal-looking cite we could find.)
IMO, soft/smooth/gradual still convey wrong impressions. They still sound like “slow takeoff”, they sound like the progress would be steady enough that normal people would have time to orient to what’s happening, keep track, and exert control. As you’re pointing out, that’s not necessarily the case at all: from a normal person’s perspective, this scenario may very much look very sharp and abrupt.
The main difference in this classification seems to be whether AI progress occurs “externally”, as part of economic and R&D ecosystems, or “internally”, as part of an opaque self-improvement process within a (set of) AI system(s). (Though IMO there’s a mostly smooth continuum of scenarios, and I don’t know that there’s a meaningful distinction/clustering at all.)
From this perspective, even continuous vs. discontinuous don’t really cleave reality at the joints. The self-improvement is still “continuous” (or, more accurately, incremental) in the hard-takeoff/RSI case, from the AI’s own perspective. It’s just that ~nothing besides the AI itself is relevant to the process.
Just “external” vs. “internal” takeoff, maybe? “Economic” vs. “unilateral”?
I do agree with that, although I don’t know that I feel the need to micromanage the implicature of the term that much.
I think it’s good to try to find terms that don’t have misleading connotations, but also good not to fight too hard to control the exact political implications of a term, partly because there’s not a clear cutoff between being clear and being actively manipulative (and not obvious to other people which you’re being, esp. if they disagree with you about the implications), and partly because there’s a bit of a red queen race of trying to get terms into common parlance that benefit your agenda, and, like, let’s just not.
Fast/slow just felt actively misleading.
I think the terms you propose here are interesting but a bit too opinionated about the mechanism involved. I’m not that confident those particular mechanisms will turn out to be decisive, and don’t think the mechanism is actually that cruxy for what the term implies in terms of strategy.
If I did want to try to give it the connotations that actually feel right to me, I might say “rolling*” as the “smooth” option. I don’t have a great “fast” one.
*although someone just said they found “rolling” unintuitive so shrug.
That is exactly the meaning that I’d thought was standard for “soft takeoff” (and which I assumed was synonymous with “slow takeoff”), e.g. as I wrote in 2012:
(B&G didn’t actually invent soft/hard takeoff, but it was the most formal-looking cite we could find.)
Assuming this is the important distinction, I like something like “isolated”/“integrated” better than either of those.