However, suppose that the box setup uses theoretically robust cybersecurity, combined with an actual physical box that is designed to not let any covert information enter or leave.
I think what you want to say here is:
However, suppose that the box setup uses robust cybersecurity, combined with an actual physical box that does not let any covert information enter or leave.
″...theoretically...” weakens rather than strengthens: we need the cybersecurity to be robust, in spite of implementation details.
It doesn’t matter what the box “is designed to” do; it matters what it does.
In both cases, this puts us in trouble: We can check the cybersecurity is theoretically robust; we can’t check it’s robust. We know what the box is designed to do; we don’t know what it does.
These holes probably aren’t too significant for superintelligent AGI (which we should expect to persuade us anyway). They may well be significant for the kinds of about-human-level systems where you’re hoping boxing might help.
On the rest, I’d largely agree with John et al. We might as well use boxing, but we shouldn’t be spending much effort on it when we could make more important progress.
I think what you want to say here is:
However, suppose that the box setup uses robust cybersecurity, combined with an actual physical box that does not let any covert information enter or leave.
″...theoretically...” weakens rather than strengthens: we need the cybersecurity to be robust, in spite of implementation details.
It doesn’t matter what the box “is designed to” do; it matters what it does.
In both cases, this puts us in trouble:
We can check the cybersecurity is theoretically robust; we can’t check it’s robust.
We know what the box is designed to do; we don’t know what it does.
These holes probably aren’t too significant for superintelligent AGI (which we should expect to persuade us anyway). They may well be significant for the kinds of about-human-level systems where you’re hoping boxing might help.
On the rest, I’d largely agree with John et al. We might as well use boxing, but we shouldn’t be spending much effort on it when we could make more important progress.