I suggest this because it is extremely simple, and in the spirit that simple solutions should be considered early rather than late. While the suggestion itself is simple, its implementation might be harder than any other approach. This is intended as a discussion, and part of what I want you to do as my reader is identify the premises which are most problematic.
My first argument aims at the conclusion that producing an AGI ‘off the assembly line’ is impossible. This is a particular case of a more general claim that for anything to be recognizable as an intelligence, it must have a familiarity with a shared world. So if we had a brain in a vat, hooked up to a simulated universe, we could only understand the brain’s activity to be thinking if we were familiar with the world the brain was thinking about. This is itself a case of the principle that in order to understand the meaning of a proposition, we have to be able to understand its truth-conditions. If I say “Schnee ist weiss” and you don’t know what I mean, I can explain the meaning of this sentence completely by pointing out that it is true if snow is white. I cannot explain its meaning without doing this.
Thus, we cannot recognize any proposition as meaningful unless we can recognize its truth conditions. And recognizing the truth conditions of a proposition requires a shared familiarity with the world on the part of the speaker and the listener. If we can’t recognize any of the propositions of a speaker as meaningful, then we can’t recognize the speaker as a speaker, nor even as a thinker. Recognizing something as intelligent requires a shared familiarity with the world.
So in order for an AGI to be recognized as intelligent, it would have to share with us a familiarity with the world. It is impossible to program this in, or in any way assemble such familiarity. It is achieved only by experience. Thus, in order to create an AGI, we would have to create a machine capable of thinking (in the way babies are) and then let it go about experiencing the world.
So while an AGI is, of course, entirely possible, programming an AGI is not. We’d have to teach it to think in the way we teach people to think.
Thus, it is of course also impossible to program an AGI to be an FAI (though it should be possible to program a potential thinker so as to make it more likely to develop friendliness). Friendliness is a way of thinking, a way of being rational, and so like all thinking it would have to be the result of experience and education. Making an AGI think, and making it friendly, are unavoidably problems of intellectual and ethical education, not in principle different from the problems of intellectual and ethical education we face with children.
Thus, the one and only way to produce an FAI is to teach it to be good in the way we teach children to be good. And if I may speak somewhat metaphorically, the problem of the singularity doesn’t seem to be radically different from the problem of having children: they will be smarter and better educated then we are, and they will produce yet smarter and even better educated children themselves, so much so that the future is opaque to us. The friendliness of children is a matter of the survival of our species, and they could easily destroy us all if they developed in a generally unfriendly way. Yet we have managed, thus far, to teach many people to be good.
My solution is thus extremely simple, and one which many, many people are presently competent to accomplish. On the other hand, trying to make one’s children into good people is more complicated and difficult, I think, than any present approach to FAI. AGI might be a greater challenge in the sense that it might be a more powerful and unruly child than any we’ve had to deal with. We might have to become better ethical teachers, and be able to teach more quickly, but the problem isn’t fundamentally different.
A Simple Solution to the FAI Problem
I suggest this because it is extremely simple, and in the spirit that simple solutions should be considered early rather than late. While the suggestion itself is simple, its implementation might be harder than any other approach. This is intended as a discussion, and part of what I want you to do as my reader is identify the premises which are most problematic.
My first argument aims at the conclusion that producing an AGI ‘off the assembly line’ is impossible. This is a particular case of a more general claim that for anything to be recognizable as an intelligence, it must have a familiarity with a shared world. So if we had a brain in a vat, hooked up to a simulated universe, we could only understand the brain’s activity to be thinking if we were familiar with the world the brain was thinking about. This is itself a case of the principle that in order to understand the meaning of a proposition, we have to be able to understand its truth-conditions. If I say “Schnee ist weiss” and you don’t know what I mean, I can explain the meaning of this sentence completely by pointing out that it is true if snow is white. I cannot explain its meaning without doing this.
Thus, we cannot recognize any proposition as meaningful unless we can recognize its truth conditions. And recognizing the truth conditions of a proposition requires a shared familiarity with the world on the part of the speaker and the listener. If we can’t recognize any of the propositions of a speaker as meaningful, then we can’t recognize the speaker as a speaker, nor even as a thinker. Recognizing something as intelligent requires a shared familiarity with the world.
So in order for an AGI to be recognized as intelligent, it would have to share with us a familiarity with the world. It is impossible to program this in, or in any way assemble such familiarity. It is achieved only by experience. Thus, in order to create an AGI, we would have to create a machine capable of thinking (in the way babies are) and then let it go about experiencing the world.
So while an AGI is, of course, entirely possible, programming an AGI is not. We’d have to teach it to think in the way we teach people to think.
Thus, it is of course also impossible to program an AGI to be an FAI (though it should be possible to program a potential thinker so as to make it more likely to develop friendliness). Friendliness is a way of thinking, a way of being rational, and so like all thinking it would have to be the result of experience and education. Making an AGI think, and making it friendly, are unavoidably problems of intellectual and ethical education, not in principle different from the problems of intellectual and ethical education we face with children.
Thus, the one and only way to produce an FAI is to teach it to be good in the way we teach children to be good. And if I may speak somewhat metaphorically, the problem of the singularity doesn’t seem to be radically different from the problem of having children: they will be smarter and better educated then we are, and they will produce yet smarter and even better educated children themselves, so much so that the future is opaque to us. The friendliness of children is a matter of the survival of our species, and they could easily destroy us all if they developed in a generally unfriendly way. Yet we have managed, thus far, to teach many people to be good.
My solution is thus extremely simple, and one which many, many people are presently competent to accomplish. On the other hand, trying to make one’s children into good people is more complicated and difficult, I think, than any present approach to FAI. AGI might be a greater challenge in the sense that it might be a more powerful and unruly child than any we’ve had to deal with. We might have to become better ethical teachers, and be able to teach more quickly, but the problem isn’t fundamentally different.