I don’t think you even need to go as far as you do here to undermine the “emergent convergence (on anti-human goals)” argument. Even if we allow that AIs, by whatever means, develop anti-human goals, what reason is there to believe that the goals (anti-human, or otherwise) of one AI would be aligned with the goals of other AIs? Although infighting among different AIs probably wouldn’t be good for humans, it is definitely not going to help AIs, as a group, in subduing humans.
Now let’s bring in something which, while left out of the primary argument, repeatedly shows up in the footnotes and counter-counter arguments: AIs need some form of human cooperation to accomplish these nefarious “goals”. Humans able to assist the AIs are a limited resource, so there is competition for them. There’s going to be a battle among the different AIs for human “mind share”.
I don’t think you even need to go as far as you do here to undermine the “emergent convergence (on anti-human goals)” argument. Even if we allow that AIs, by whatever means, develop anti-human goals, what reason is there to believe that the goals (anti-human, or otherwise) of one AI would be aligned with the goals of other AIs? Although infighting among different AIs probably wouldn’t be good for humans, it is definitely not going to help AIs, as a group, in subduing humans.
Now let’s bring in something which, while left out of the primary argument, repeatedly shows up in the footnotes and counter-counter arguments: AIs need some form of human cooperation to accomplish these nefarious “goals”. Humans able to assist the AIs are a limited resource, so there is competition for them. There’s going to be a battle among the different AIs for human “mind share”.