I see your point, but Voldemort hasn’t encountered the AI Box problem has he? Further, I don’t think Voldemort has encountered a problem where he’s arguing with someone/something he knows is far smarter than himself. He still believes Harry isn’t as smart yet.
6) Harry is playing against the intelligent but naïve Voldemort instead of against the intelligent and experienced Nathan Russell. (Actually, I don’t know who Russell is apart from being the first person to let EY out of the box, but he may well be experienced with this problem, for all I know, and he’s probably intelligent if he got into this stuff at all.)
Other than (5), these are all things that are liable to be true of an AI asking to be let out of the box.
Code that appears Friendly but has not been proved Friendly
Advanced intelligence of the AI
General programming goals, much weaker than (1) really
True verbatim in the standard AI box experiment (and arguably in the real world right now)
I see your point, but Voldemort hasn’t encountered the AI Box problem has he? Further, I don’t think Voldemort has encountered a problem where he’s arguing with someone/something he knows is far smarter than himself. He still believes Harry isn’t as smart yet.
Sure, but now your argument, it seems to me, is
6) Harry is playing against the intelligent but naïve Voldemort instead of against the intelligent and experienced Nathan Russell. (Actually, I don’t know who Russell is apart from being the first person to let EY out of the box, but he may well be experienced with this problem, for all I know, and he’s probably intelligent if he got into this stuff at all.)