I’m more worried that a term like “safe AGI” could provoke a response of “So you’re trying to make sure that a system which is smarter than humans, and able to operate in arbitrary real-world environments, and able to invent new technologies to achieve its goals, will be safe? Let me save you some time and tell you right now that’s impossible. Your research program is a pipe dream.”
If someone has this reaction, then can’t you just say “mission accomplished” and not worry about it too much? In any case, I think “AI safety” is probably the most beneficial to your goals. I would also not be too worried about AI researchers having a knee-jerk response to the term, for the same reasons you do.
I agree. “AGI Safety”/”Safe AGI” seems like the best option. if people say, “Let me save you some time and tell you right now that’s impossible” half of the work is done. The other half is just convincing them that we have to do it anyway because otherwise everyone is doomed. (This is of course, as long as they are using “impossible” in a loose sense. If they aren’t, the problem can probably be fixed by saying “our definition of safety is a little bit more loose than the one you’re probably thinking of, but not so much more loose that it becomes easy”).
Yeah, my kneejerk reaction to someone saying “Fat chance, that’s impossible.” is to retort “Should we be trying to make an unsafe AI and hoping to reap the benefits of our superintelligence even if it’s not guaranteed to destroy us all, or should we be trying to stop everyone else in the entire world from doing that? Because that seems just as impossible.”
If someone has this reaction, then can’t you just say “mission accomplished” and not worry about it too much? In any case, I think “AI safety” is probably the most beneficial to your goals. I would also not be too worried about AI researchers having a knee-jerk response to the term, for the same reasons you do.
I agree. “AGI Safety”/”Safe AGI” seems like the best option. if people say, “Let me save you some time and tell you right now that’s impossible” half of the work is done. The other half is just convincing them that we have to do it anyway because otherwise everyone is doomed. (This is of course, as long as they are using “impossible” in a loose sense. If they aren’t, the problem can probably be fixed by saying “our definition of safety is a little bit more loose than the one you’re probably thinking of, but not so much more loose that it becomes easy”).
Yeah, my kneejerk reaction to someone saying “Fat chance, that’s impossible.” is to retort “Should we be trying to make an unsafe AI and hoping to reap the benefits of our superintelligence even if it’s not guaranteed to destroy us all, or should we be trying to stop everyone else in the entire world from doing that? Because that seems just as impossible.”