“Do you actually believe that it is possible for a mere human being to ever be 100% certain that a given AGI design will not lead to the destruction of humanity?”
Well, obviously one can’t be 100% certain, but I’d be curious to know exactly how certain Eliezer wants to be before he presses the start button on his putative FAI. 99.9%? 99.99%? And, Samantha, what’s your cutoff for reasonable certainty in this situation? 90%? 99%?
“I can’t buy into the notion that careful design of initial conditions of the AGI and of its starting learning algorithms are sufficient for the guarantee you seem to seek.”
This is the same way I understand him, and I also think it’s pretty audacious, but just maybe possible. I’m vaguely familiar with some of the techniques you might use to go about doing this, and it seems like a really hard problem, but not impossible.
“I also don’t get why “I need to beat my competitors” is even remotely a consideration”
How about “I need to beat my non-FAI-savvy competitors”?
“Do you actually believe that it is possible for a mere human being to ever be 100% certain that a given AGI design will not lead to the destruction of humanity?”
Well, obviously one can’t be 100% certain, but I’d be curious to know exactly how certain Eliezer wants to be before he presses the start button on his putative FAI. 99.9%? 99.99%? And, Samantha, what’s your cutoff for reasonable certainty in this situation? 90%? 99%?
“I can’t buy into the notion that careful design of initial conditions of the AGI and of its starting learning algorithms are sufficient for the guarantee you seem to seek.”
This is the same way I understand him, and I also think it’s pretty audacious, but just maybe possible. I’m vaguely familiar with some of the techniques you might use to go about doing this, and it seems like a really hard problem, but not impossible.
“I also don’t get why “I need to beat my competitors” is even remotely a consideration”
How about “I need to beat my non-FAI-savvy competitors”?