An AI that is indistinguishable from a human (to an even greater superintelligent AI) is not dangerous, because humans are not dangerous. Just like a chess master that is indistinguishable from a regular player wouldn’t win many games.
It may be indistinguishable until it gets our of the building. Recent movie Ex Machine had such plot.
The AI doesn’t want to escape from the building. It’s utility function is basically to mimic humans. It’s a terminal value, not a subgoal.
But most humans would like to escape from any confinement
An AI that is indistinguishable from a human (to an even greater superintelligent AI) is not dangerous, because humans are not dangerous. Just like a chess master that is indistinguishable from a regular player wouldn’t win many games.
It may be indistinguishable until it gets our of the building. Recent movie Ex Machine had such plot.
The AI doesn’t want to escape from the building. It’s utility function is basically to mimic humans. It’s a terminal value, not a subgoal.
But most humans would like to escape from any confinement