Something just hit me. Maybe I am misremembering, but I don’t recall hearing the idea of an intelligence explosion being explained using compound interest as an analogy. And it seems like a very useful and intuitive analogy.
I was just thinking about it being analogous to productivity. Imagine your productivity is a 4⁄10. Say you focus on improving. Now you’re at a 6⁄10. Ok, now say you want to continue to improve your productivity. Now you’re at a 6⁄10, so you are more capable of increasing your productivity further. So now instead of increasing two points to an 8⁄10, you are capable of increasing, say, three points, to a 9⁄10. Then I run into the issue with the “out of ten” part setting a ceiling on how high it can go. But it is awkward to describe productivity as “a four”. So I was thinking about how to phrase it.
But then I realized it’s like compound interest! Say you start with four dollars. You have access to something that grows your money at 50% interest (lucky you), so it grows to six dollars. But now you have six dollars, and still have access to that think that grows at 50%. 50% of six is three, so you go from six to nine dollars. And then now that you have nine, you have an even bigger thing to take 50% of. Etc. etc. Intelligence exploding seems very analogous. The more intelligence you have, the easier it is to grow it.
I never studied this stuff formally though. Just absorbed some stuff by hanging out around LessWrong over time. So someone please correct me if I am misunderstanding stuff.
I think Eliezer’s original analogy (which may or may not be right, but is a fun thing to think about mathematically) was more like “compound interest folded on itself”. Imagine you’re a researcher making progress at a fixed rate, improving computers by 10% per year. That’s modeled well by compound interest, since every year there’s a larger number to increase by 10%, and it gives your ordinary exponential curve. But now make an extra twist: imagine the computing advances are speeding up your research as well, maybe because your mind is running on a computer, or because of some less exotic effects. So the first 10% improvement happens after a year, the next after 11 months, and so on. This may not be obvious, but it changes the picture qualitatively: it gives not just a faster exponential, but a curve which has a vertical asymptote, going to infinity in finite time. The reason is that the descending geometrical progression—a year, plus 11 months, and so on—adds up to a finite amount of time, in the same way that 1+1/2+1/4… adds up to a finite amount.
Of course there’s no infinity in real life, but the point is that a situation where research makes research faster could be even more unstable (“gradual and then sudden”) than ordinary compound interest, which we already have trouble understanding intuitively.
Something just hit me. Maybe I am misremembering, but I don’t recall hearing the idea of an intelligence explosion being explained using compound interest as an analogy. And it seems like a very useful and intuitive analogy.
I was just thinking about it being analogous to productivity. Imagine your productivity is a 4⁄10. Say you focus on improving. Now you’re at a 6⁄10. Ok, now say you want to continue to improve your productivity. Now you’re at a 6⁄10, so you are more capable of increasing your productivity further. So now instead of increasing two points to an 8⁄10, you are capable of increasing, say, three points, to a 9⁄10. Then I run into the issue with the “out of ten” part setting a ceiling on how high it can go. But it is awkward to describe productivity as “a four”. So I was thinking about how to phrase it.
But then I realized it’s like compound interest! Say you start with four dollars. You have access to something that grows your money at 50% interest (lucky you), so it grows to six dollars. But now you have six dollars, and still have access to that think that grows at 50%. 50% of six is three, so you go from six to nine dollars. And then now that you have nine, you have an even bigger thing to take 50% of. Etc. etc. Intelligence exploding seems very analogous. The more intelligence you have, the easier it is to grow it.
I never studied this stuff formally though. Just absorbed some stuff by hanging out around LessWrong over time. So someone please correct me if I am misunderstanding stuff.
I think Eliezer’s original analogy (which may or may not be right, but is a fun thing to think about mathematically) was more like “compound interest folded on itself”. Imagine you’re a researcher making progress at a fixed rate, improving computers by 10% per year. That’s modeled well by compound interest, since every year there’s a larger number to increase by 10%, and it gives your ordinary exponential curve. But now make an extra twist: imagine the computing advances are speeding up your research as well, maybe because your mind is running on a computer, or because of some less exotic effects. So the first 10% improvement happens after a year, the next after 11 months, and so on. This may not be obvious, but it changes the picture qualitatively: it gives not just a faster exponential, but a curve which has a vertical asymptote, going to infinity in finite time. The reason is that the descending geometrical progression—a year, plus 11 months, and so on—adds up to a finite amount of time, in the same way that 1+1/2+1/4… adds up to a finite amount.
Of course there’s no infinity in real life, but the point is that a situation where research makes research faster could be even more unstable (“gradual and then sudden”) than ordinary compound interest, which we already have trouble understanding intuitively.