It’s an effective response to talk like “But why not work on a maybe-Friendly AI, it’s better than nothing” that I don’t usually see.
It’s a generally useful insight, that even if we can employ a mathematical proof, we only have a “Proven Friendly AI with N% confidence” for some N, and so a well-considered 1% FAI is still a FAI, since the default is ”?”. Generally useful as in, that insight applies to practically everything else.
Edited quote.
It’s an effective response to talk like “But why not work on a maybe-Friendly AI, it’s better than nothing” that I don’t usually see.
It’s a generally useful insight, that even if we can employ a mathematical proof, we only have a “Proven Friendly AI with N% confidence” for some N, and so a well-considered 1% FAI is still a FAI, since the default is ”?”. Generally useful as in, that insight applies to practically everything else.