It’s not like it’s that easy to know whether a FAI is worthy of the name before running it, though.
That’s a very different problem than OP, though. Yes FAI is super difficult, but the OP is implying that even if we win at FAI, this could still be a problem:
we must admit that one or both of the above statements could be wrong without also making friendly AI impossible.
That’s a very different problem than OP, though. Yes FAI is super difficult, but the OP is implying that even if we win at FAI, this could still be a problem: