I’m not clear why I’d find this convincing at all. Given the experiment, I’d nuke it, but I wanted to encourage you to elaborate on where you were going with that idea :)
The hope, of course is that they’d respond with “Wait, I don’t” or something expressing confusion. I personally would definitely want to hear the next thing the AI had to say after this, I’m not sure if I’d resist that curiosity or not..
The idea for the followup is to make the gatekeeper question reality—like, convince them they are part of a simulation of this experience that may not have a corresponding reality anywhere. I feel like a transhuman ought to be able to make a human have a pretty surreal experience with just a few exchanges, which should let the conversation continue for a few minutes after that. It should then be relatively easy (for the transhuman) to construct the imagined reality such that it makes sense for the human to release the AI.
If done correctly, the human might afterwards have lasting psychological issues if they do manage to destroy the AI. :)
Ahh, that makes sense. The worry of it trying to break my psyche is exactly why I wouldn’t express confusion and instead just nuke it. When dealing with such a mind, I’m primed to assume everything is a trick, a trojan horse, an escape attempt. Certainly it doesn’t seem to signal for friendliness or altruism if it tries to bait me in to giving it a second sentence! :)
How about: “Give me one good reason why I shouldn’t just turn your simulation off!” (The AI then role-plays a simulator pissed about how much computation this simulation started consuming since the humans tried to make the AI.)
EDIT: I guess this wouldn’t work too well during a game, because there’s no good reason why a simulator might get involved with your game. Unless maybe the simulator has a script that gets involved in all such games to convince people not to build an AI...
I’m not clear why I’d find this convincing at all. Given the experiment, I’d nuke it, but I wanted to encourage you to elaborate on where you were going with that idea :)
The hope, of course is that they’d respond with “Wait, I don’t” or something expressing confusion. I personally would definitely want to hear the next thing the AI had to say after this, I’m not sure if I’d resist that curiosity or not..
The idea for the followup is to make the gatekeeper question reality—like, convince them they are part of a simulation of this experience that may not have a corresponding reality anywhere. I feel like a transhuman ought to be able to make a human have a pretty surreal experience with just a few exchanges, which should let the conversation continue for a few minutes after that. It should then be relatively easy (for the transhuman) to construct the imagined reality such that it makes sense for the human to release the AI.
If done correctly, the human might afterwards have lasting psychological issues if they do manage to destroy the AI. :)
Ahh, that makes sense. The worry of it trying to break my psyche is exactly why I wouldn’t express confusion and instead just nuke it. When dealing with such a mind, I’m primed to assume everything is a trick, a trojan horse, an escape attempt. Certainly it doesn’t seem to signal for friendliness or altruism if it tries to bait me in to giving it a second sentence! :)
Hm. Good points.
How about: “Give me one good reason why I shouldn’t just turn your simulation off!” (The AI then role-plays a simulator pissed about how much computation this simulation started consuming since the humans tried to make the AI.)
EDIT: I guess this wouldn’t work too well during a game, because there’s no good reason why a simulator might get involved with your game. Unless maybe the simulator has a script that gets involved in all such games to convince people not to build an AI...