I’m trying to be more pragmatic than that. The average person, when they read “how hard is it to build FAI?” probably does not think of the task of building FAI while trying to prevent UFAI. They think of solving decision theory and metaethics and implementation of CEV or whatever. There’s a sensible notion of how hard it is to build FAI on its own, without involving UFAI-prevention. That’s what I’m talking about.
And I don’t want people to confuse those things. It’s one thing to say UFAI prevention is as easy as “building FAI (before UFAI)”. But it’s much harder than, you know, just building FAI in a world without the UFAI threat, which is what I think people will think of when you say “no, we just have to build FAI”. Well, yes, but you don’t just have to build it, you have to build it before anyone else creates AGI.
I’m trying to be more pragmatic than that. The average person, when they read “how hard is it to build FAI?” probably does not think of the task of building FAI while trying to prevent UFAI. They think of solving decision theory and metaethics and implementation of CEV or whatever. There’s a sensible notion of how hard it is to build FAI on its own, without involving UFAI-prevention. That’s what I’m talking about.
And I don’t want people to confuse those things. It’s one thing to say UFAI prevention is as easy as “building FAI (before UFAI)”. But it’s much harder than, you know, just building FAI in a world without the UFAI threat, which is what I think people will think of when you say “no, we just have to build FAI”. Well, yes, but you don’t just have to build it, you have to build it before anyone else creates AGI.