The “know” being in italics and the following “(maybe not a very good one, but still)” are meant to stress that “maybe it’ll work, dunno” is not an intended interpretation.
It’s an effective response to talk like “But why not work on a maybe-Friendly AI, it’s better than nothing” that I don’t usually see.
It’s a generally useful insight, that even if we can employ a mathematical proof, we only have a “Proven Friendly AI with N% confidence” for some N, and so a well-considered 1% FAI is still a FAI, since the default is ”?”. Generally useful as in, that insight applies to practically everything else.
The “know” being in italics and the following “(maybe not a very good one, but still)” are meant to stress that “maybe it’ll work, dunno” is not an intended interpretation.
Edited quote.
It’s an effective response to talk like “But why not work on a maybe-Friendly AI, it’s better than nothing” that I don’t usually see.
It’s a generally useful insight, that even if we can employ a mathematical proof, we only have a “Proven Friendly AI with N% confidence” for some N, and so a well-considered 1% FAI is still a FAI, since the default is ”?”. Generally useful as in, that insight applies to practically everything else.