This should probably go on a FAI FAQ, especially this bit:
If you have a plan for which you know that it has some chance of success (say, above 1%), you have a design of FAI (maybe not a very good one, but still).
The “know” being in italics and the following “(maybe not a very good one, but still)” are meant to stress that “maybe it’ll work, dunno” is not an intended interpretation.
It’s an effective response to talk like “But why not work on a maybe-Friendly AI, it’s better than nothing” that I don’t usually see.
It’s a generally useful insight, that even if we can employ a mathematical proof, we only have a “Proven Friendly AI with N% confidence” for some N, and so a well-considered 1% FAI is still a FAI, since the default is ”?”. Generally useful as in, that insight applies to practically everything else.
This should probably go on a FAI FAQ, especially this bit:
The “know” being in italics and the following “(maybe not a very good one, but still)” are meant to stress that “maybe it’ll work, dunno” is not an intended interpretation.
Edited quote.
It’s an effective response to talk like “But why not work on a maybe-Friendly AI, it’s better than nothing” that I don’t usually see.
It’s a generally useful insight, that even if we can employ a mathematical proof, we only have a “Proven Friendly AI with N% confidence” for some N, and so a well-considered 1% FAI is still a FAI, since the default is ”?”. Generally useful as in, that insight applies to practically everything else.