This is why I don’t like “AI safety”. It’s implicitly setting the bar too low. “Friendliness” in theory has the same problem, but Eliezer actually seems to be aiming at “ideal”, while working under that name. When Luke asked for suggestions for how to rename their research program, I suggested “optimal AI”, but he didn’t seem to like that very much.
This is why I don’t like “AI safety”. It’s implicitly setting the bar too low. “Friendliness” in theory has the same problem, but Eliezer actually seems to be aiming at “ideal”, while working under that name. When Luke asked for suggestions for how to rename their research program, I suggested “optimal AI”, but he didn’t seem to like that very much.
How about BAI for Best AI?
This is actually more or less what I am getting at when I talk about distinctions between FAI and Obedient AI.