It is sometimes argued that even if this advantage is modest, the growth curves will be exponential, and therefore a slight advantage right now will compound to become a large advantage over a long enough period of time. However, this argument by itself is not an argument against a continuous takeoff.
I’m not sure this is an accurate characterization of the point; my understanding is that the concern largely comes from the possibility that the growth will be faster than exponential, rather than merely exponential.
I’m not sure this is an accurate characterization of the point; my understanding is that the concern largely comes from the possibility that the growth will be faster than exponential
Sure, if someone was arguing that, then they have a valid understanding of the difference between continuous vs. discontinuous takeoff. I would just question the assumption why we should expect growth to be faster than exponential for any sustained period of time.
To be clear, intelligence explosion via recursive self-improvement has been distinguished from merely exponential growth at least as far back as Yudkowsky’s “Three Major Singularity Schools”. I couldn’t remember the particular link when I wrote the comment above, but, well, now I remember it.
Anyway, I don’t have a particular argument one way or the other; I’m just registering my surprise that you encountered people here arguing for merely exponential growth base on intelligence explosion arguments.
Empirically, most systems with a feedback loop don’t grow hyperbolically. I would need strong theoretical reasons in order to understand why this particular distinction is important.
I don’t really want to go trying to defend here a position I don’t necessarily hold, but I do have to nitpick and point out that there’s quite a bit of room inbetween exponential and hyperbolic.
I’m not sure this is an accurate characterization of the point; my understanding is that the concern largely comes from the possibility that the growth will be faster than exponential, rather than merely exponential.
Sure, if someone was arguing that, then they have a valid understanding of the difference between continuous vs. discontinuous takeoff. I would just question the assumption why we should expect growth to be faster than exponential for any sustained period of time.
To be clear, intelligence explosion via recursive self-improvement has been distinguished from merely exponential growth at least as far back as Yudkowsky’s “Three Major Singularity Schools”. I couldn’t remember the particular link when I wrote the comment above, but, well, now I remember it.
Anyway, I don’t have a particular argument one way or the other; I’m just registering my surprise that you encountered people here arguing for merely exponential growth base on intelligence explosion arguments.
Empirically, most systems with a feedback loop don’t grow hyperbolically. I would need strong theoretical reasons in order to understand why this particular distinction is important.
I don’t really want to go trying to defend here a position I don’t necessarily hold, but I do have to nitpick and point out that there’s quite a bit of room inbetween exponential and hyperbolic.