I don’t think you get the same thing from that document that I do. (Incidentally, I disagree with a lot of the design decisions inherent in that document, such as self-modifying AI, which I regard as inherently and uncorrectably dangerous. When you stop expecting the AI to make itself better, the “Keep your ethics stable across iterations” part of the problem goes away.)
Either that or I’m misunderstanding you. Because my current understanding of your view of the Friendliness problem has less to do with codifying and programming ethics and more to do with teaching the AI to know exactly what we mean and not to misinterpret what we ask for. (Which I hope you’ll forgive me if I call “Magical thinking.” That’s not necessarily a disparagement; sufficiently advanced technology and all that. I just think it’s not feasible in the foreseeable future, and such an AI makes a poor target for us as we exist today.)
I don’t think you get the same thing from that document that I do. (Incidentally, I disagree with a lot of the design decisions inherent in that document, such as self-modifying AI, which I regard as inherently and uncorrectably dangerous. When you stop expecting the AI to make itself better, the “Keep your ethics stable across iterations” part of the problem goes away.)
Either that or I’m misunderstanding you. Because my current understanding of your view of the Friendliness problem has less to do with codifying and programming ethics and more to do with teaching the AI to know exactly what we mean and not to misinterpret what we ask for. (Which I hope you’ll forgive me if I call “Magical thinking.” That’s not necessarily a disparagement; sufficiently advanced technology and all that. I just think it’s not feasible in the foreseeable future, and such an AI makes a poor target for us as we exist today.)