design the AI in such a way that it can create agents, but only
This sort of argument would be much more valuable if accompanied by a specific recipe of how to do it, or at least a proof that one must exist. Why worry about AI designing agents, why not just “design it in such a way” that it’s already Friendly!
I agree. I didn’t mean to imply that I thought this step would be easy, and I would also be interested in more concrete ways of doing it. It’s possible that creating a hereditarily restricted optimizer along the lines I was suggesting could end up being approximately as difficult as creating an aligned general-purpose optimizer, but I intuitively don’t expect this to be the case.
This sort of argument would be much more valuable if accompanied by a specific recipe of how to do it, or at least a proof that one must exist. Why worry about AI designing agents, why not just “design it in such a way” that it’s already Friendly!
I agree. I didn’t mean to imply that I thought this step would be easy, and I would also be interested in more concrete ways of doing it. It’s possible that creating a hereditarily restricted optimizer along the lines I was suggesting could end up being approximately as difficult as creating an aligned general-purpose optimizer, but I intuitively don’t expect this to be the case.