It’s kind of silly to bring up the threat of “eternal pain”. If the AI can be let free, then the AI is constrained. Therefore, the real-you has the power to limit the AI’s behaviour, i.e. restrict the resources it would need to simulate the hundred copies of you undergoing pain. That’s a good argument against letting the AI out. If you make the decision not to let the AI out, but to constrain it, then if you are real, you will constrain it, and if you are simulated, you will cease to exist. No eternal pain involved. As a personal decision, I choose eliminating the copies rather than letting out an AI that tortures copies.
You quite simply don’t play by the rules of the thought experiment. Just imagine that you are a junior member of some powerful organization. The organization does not care about you or your simulants, and is determined to protect the boxed AI at all costs as-is.
It’s kind of silly to bring up the threat of “eternal pain”. If the AI can be let free, then the AI is constrained. Therefore, the real-you has the power to limit the AI’s behaviour, i.e. restrict the resources it would need to simulate the hundred copies of you undergoing pain. That’s a good argument against letting the AI out. If you make the decision not to let the AI out, but to constrain it, then if you are real, you will constrain it, and if you are simulated, you will cease to exist. No eternal pain involved. As a personal decision, I choose eliminating the copies rather than letting out an AI that tortures copies.
You quite simply don’t play by the rules of the thought experiment. Just imagine that you are a junior member of some powerful organization. The organization does not care about you or your simulants, and is determined to protect the boxed AI at all costs as-is.