Maybe they develop mind control level convincing argument and send it to key people (president, congress, NORAD, etc) or hack their iPhones and recursively down to security guards of fabs/power plants/data centers/drone factories. That may be quick enough. The point is that it is not obvious.
That’s the sort of thing that’d happen, yes. As with all AI takeover scenarios, it likely wouldn’t go down like this specifically, but you can be sure that the ASI would achieve the goal it wants to achieve/was told to achieve if aligned. (And see this post for my model of how this class of concrete scenarios would actually look like.)
Having nukes is not really a good analogy for having an aligned ASI at your disposal, as far as taking over the world is concerned. Unless your terminal value is human extinction, you can’t really nuke the world into the state of your personal utopia. You can’t even use nukes as leverage to threaten people into building your utopia, because:
Some people are good enough at decision theory to ignore threats.
Coercing people in this way might not actually be part of your utopia.
Your “power” is brittle. You only have the threat of nuclear armageddon to fall back on, and you can still be defeated by e. g. clever infiltration and sabotage, or by taking over your supply chains, etc. (If you have overwhelming, utterly loyal military power and security in full generality, that’s a very different setup.)
None of those constraints apply to having an ASI at your disposal. An ASI would let you implement your values upon the cosmos fully and faithfully, and it’d give you the roadmap to getting there from here.
This is also precisely why Leopold’s talk of “checks and balances” as the reason why governments could be trusted with AGI falls apart. “The government” isn’t some sort of holistic entity, it’s a collection of individuals with their own incentives, sometimes quite monstrous incentives. In the current regime, it’s indeed checked-and-balanced to be mostly sort-of (not really) aligned to the public good. But that property is absolutely not robust to you giving unchecked power to any given subsystem in it!
I’m really quite baffled that Leopold doesn’t get this, given his otherwise excellent analysis of the “authoritarianism risks” associated with aligned ASIs in the hands of private companies and the CCP. Glad to see @Zvi pointing that out.
That’s the sort of thing that’d happen, yes. As with all AI takeover scenarios, it likely wouldn’t go down like this specifically, but you can be sure that the ASI would achieve the goal it wants to achieve/was told to achieve if aligned. (And see this post for my model of how this class of concrete scenarios would actually look like.)
Having nukes is not really a good analogy for having an aligned ASI at your disposal, as far as taking over the world is concerned. Unless your terminal value is human extinction, you can’t really nuke the world into the state of your personal utopia. You can’t even use nukes as leverage to threaten people into building your utopia, because:
Some people are good enough at decision theory to ignore threats.
Coercing people in this way might not actually be part of your utopia.
Your “power” is brittle. You only have the threat of nuclear armageddon to fall back on, and you can still be defeated by e. g. clever infiltration and sabotage, or by taking over your supply chains, etc. (If you have overwhelming, utterly loyal military power and security in full generality, that’s a very different setup.)
None of those constraints apply to having an ASI at your disposal. An ASI would let you implement your values upon the cosmos fully and faithfully, and it’d give you the roadmap to getting there from here.
This is also precisely why Leopold’s talk of “checks and balances” as the reason why governments could be trusted with AGI falls apart. “The government” isn’t some sort of holistic entity, it’s a collection of individuals with their own incentives, sometimes quite monstrous incentives. In the current regime, it’s indeed checked-and-balanced to be mostly sort-of (not really) aligned to the public good. But that property is absolutely not robust to you giving unchecked power to any given subsystem in it!
I’m really quite baffled that Leopold doesn’t get this, given his otherwise excellent analysis of the “authoritarianism risks” associated with aligned ASIs in the hands of private companies and the CCP. Glad to see @Zvi pointing that out.