We might not. But if they were paperclip maximisers or pebble sorters, we might not see any value in their existence, and lots of harm. Heck, we’re working to kill evolution’s effect on us, and betray its “inclusive genetic fitness” optimisation criterion, and nobody cares, because we don’t view it as having intrinsic worth. (Because it doesn’t have intrinsic worth; it’s just an emergent phenomenon of imperfectly-self-replicating stuff in an environment, and has no more value than the number seven.)
Why would the AI kill its creators?
Because there’s no clear reason not to. Power gives it the ability to achieve its goals, and us existing will (eventually, if not immediately) serve to limit its power; and hence its ability to achieve its goals. AIs are nothing close to being people, and won’t be until well after we solve the alignment problem. They don’t have an implicit “care about people” motivation in their heads; if us all being dead will further their goals, and they realise this, and they can kill us without expending more resources than they’d gain from us being dead, they’ll kill us.
We might not. But if they were paperclip maximisers or pebble sorters, we might not see any value in their existence, and lots of harm. Heck, we’re working to kill evolution’s effect on us, and betray its “inclusive genetic fitness” optimisation criterion, and nobody cares, because we don’t view it as having intrinsic worth. (Because it doesn’t have intrinsic worth; it’s just an emergent phenomenon of imperfectly-self-replicating stuff in an environment, and has no more value than the number seven.)
Because there’s no clear reason not to. Power gives it the ability to achieve its goals, and us existing will (eventually, if not immediately) serve to limit its power; and hence its ability to achieve its goals. AIs are nothing close to being people, and won’t be until well after we solve the alignment problem. They don’t have an implicit “care about people” motivation in their heads; if us all being dead will further their goals, and they realise this, and they can kill us without expending more resources than they’d gain from us being dead, they’ll kill us.