If you decide to write that post, it would be great if you started by describing the potential impact of metaethics on FAI design, to make sure that we’re answering questions that need answering and aren’t just confusions about words. If anyone wants to take a stab here in the comments, I’d be very interested.
Well… according to the SEP, metaethics encompasses the attempt to understand the presuppositions and commitments of moral practice.
If I’m trying to engineer a system that behaves morally (which is what FAI design is, right?), it makes some sense that I’d want to understand that stuff, just as if I’m trying to engineer a system that excavates tunnels I’d want to understand the presuppositions and commitments of tunnel excavation.
That said, from what I’ve seen it’s not clear to me that the actual work that’s been done in this area (e.g., in the Metaethics Sequence) actually serves any purpose other than rhetorical.
I think that framing the issue of AI safety in terms of “morality” or “friendliness” is a form of misleading anthropomorphization. Morality and friendliness are specific traits of human psychology which won’t necessarily generalize well to artificial agents (even attempts to generalize them to non-human animals are often far-fetched). I think that AI safety would be probably best dealt with in the framework of safety engineering.
All right. I certainly agree with you that talking about “morality” or “friendliness” without additional clarifications leads most people to conclusions that have very little to do with safe AI design. Then again, if we’re talking about self-improving AIs with superhuman intelligence (as many people on this site are) I think the same is true of talking about “safety.”
If you decide to write that post, it would be great if you started by describing the potential impact of metaethics on FAI design, to make sure that we’re answering questions that need answering and aren’t just confusions about words. If anyone wants to take a stab here in the comments, I’d be very interested.
Well… according to the SEP, metaethics encompasses the attempt to understand the presuppositions and commitments of moral practice.
If I’m trying to engineer a system that behaves morally (which is what FAI design is, right?), it makes some sense that I’d want to understand that stuff, just as if I’m trying to engineer a system that excavates tunnels I’d want to understand the presuppositions and commitments of tunnel excavation.
That said, from what I’ve seen it’s not clear to me that the actual work that’s been done in this area (e.g., in the Metaethics Sequence) actually serves any purpose other than rhetorical.
I think that framing the issue of AI safety in terms of “morality” or “friendliness” is a form of misleading anthropomorphization. Morality and friendliness are specific traits of human psychology which won’t necessarily generalize well to artificial agents (even attempts to generalize them to non-human animals are often far-fetched).
I think that AI safety would be probably best dealt with in the framework of safety engineering.
All right. I certainly agree with you that talking about “morality” or “friendliness” without additional clarifications leads most people to conclusions that have very little to do with safe AI design. Then again, if we’re talking about self-improving AIs with superhuman intelligence (as many people on this site are) I think the same is true of talking about “safety.”