Sorry, I think I am guilty of misusing terminology. I have been using AI and AGI interchangeably, but that’s obviously not right. As far as I understand, “AGI” refers to a general intelligence who can solve (or, at least, attempt to solve) any problem, whereas “AI” refers to any kind of an artificial intelligence, including the specialized kind. There are many AIs that already exist in the world—for example, Google’s AdSense algorithm—but SIAI is not concerned about them (as far as I know), because they lack the capacity to self-improve.
My own hidden assumption, which I should’ve recognized and voiced earlier, is that an AGI (as contrasted with non-general AI) would most likely be produced through a process of recursive self-improvement; it is highly unlikely that an AGI could be created from scratch by humans writing lines of code. As far as I understand, the SIAI agrees with this statement, but again, I could be wrong.
Thus, it is unlikely that a non-general AI will ever be smart enough to warrant concern. It could still do some damage, of course, but then, so could a busted water main. On the other hand, an AGI will most likely arise as the result of recursive self-improvement, and thus will be capable of further self-improvement, thus boosting itself to transhuman levels very quickly unless its self-improvement is arrested by some mechanism.
Yeah, I’ve been talking throughout about what you’re labeling “AI” here. We agree that these won’t necessarily self-improve. Awesome.
With respect to what you’re labeling “AGI” here, you’re saying the following: 1) given that X is an AGI developed by humans, the probability that X has thus far been capable of recursive self-improvement is very high, and 2) given that X has thus far been capable of recursive self-improvement, the probability that X will continue to be capable of recursive self-improvement in the future is very high. 3) SIAI believes 1) and 2).
Yes, with the caveats that a). as far as I know, no such X currently exists, and b). my confidence in (3) is much lower than my confidence in (1) and (2).
Sorry, I think I am guilty of misusing terminology. I have been using AI and AGI interchangeably, but that’s obviously not right. As far as I understand, “AGI” refers to a general intelligence who can solve (or, at least, attempt to solve) any problem, whereas “AI” refers to any kind of an artificial intelligence, including the specialized kind. There are many AIs that already exist in the world—for example, Google’s AdSense algorithm—but SIAI is not concerned about them (as far as I know), because they lack the capacity to self-improve.
My own hidden assumption, which I should’ve recognized and voiced earlier, is that an AGI (as contrasted with non-general AI) would most likely be produced through a process of recursive self-improvement; it is highly unlikely that an AGI could be created from scratch by humans writing lines of code. As far as I understand, the SIAI agrees with this statement, but again, I could be wrong.
Thus, it is unlikely that a non-general AI will ever be smart enough to warrant concern. It could still do some damage, of course, but then, so could a busted water main. On the other hand, an AGI will most likely arise as the result of recursive self-improvement, and thus will be capable of further self-improvement, thus boosting itself to transhuman levels very quickly unless its self-improvement is arrested by some mechanism.
OK, I think I understand better now.
Yeah, I’ve been talking throughout about what you’re labeling “AI” here. We agree that these won’t necessarily self-improve. Awesome.
With respect to what you’re labeling “AGI” here, you’re saying the following:
1) given that X is an AGI developed by humans, the probability that X has thus far been capable of recursive self-improvement is very high, and
2) given that X has thus far been capable of recursive self-improvement, the probability that X will continue to be capable of recursive self-improvement in the future is very high.
3) SIAI believes 1) and 2).
Yes? Have I understood you?
Yes, with the caveats that a). as far as I know, no such X currently exists, and b). my confidence in (3) is much lower than my confidence in (1) and (2).