I think most everyone at MIRI and FHI thinks boxing is a good thing, even if many would say not enough on its own. I don’t think you will find many who think that open internet connections are a matter of indifference for AI developers working with powerful AGI.
High-grade common sense (the sort you’d get by asking any specialist in computer security) says that you should design an AI which you would trust with an open Internet connection, then put it in the box you would use on an untrusted AI during development. (No, the AI will not be angered by this lack of trust and resent you. Thank you for asking.) I think it’s safe to say that for basically everything in FAI strategy (I can’t think of an exception right now) you can identify at least two things supporting any key point, such that either alone was designed to be sufficient independently of the other’s failing, including things like “indirect normativity works” (you try to build in at least some human checks around this which would shut down any scary AI independently of your theory of indirect normativity being remotely correct, while also not trusting the humans to steer the AI because then the humans are your single point of failure).
But didn’t you say that if you find that your FAI design needs to be patched up to resolve some issue, you should rather abandon it and look for another? This redundancy of security looks like a patchup to me. As long as neither the AIs intent nor the boxing is proven to be secure, we are not ready to start development; once one is, we do not need the other.
Note also that Roman co-authored 3 of the papers on MIRI’s publications page.
His paper http://cecs.louisville.edu/ry/LeakproofingtheSingularity.pdf seriously discusses ways to confine a potentially hostile superintelligence, a feat MIRI seems to consider hopeless. Did you guys have a good chat about it?
I think most everyone at MIRI and FHI thinks boxing is a good thing, even if many would say not enough on its own. I don’t think you will find many who think that open internet connections are a matter of indifference for AI developers working with powerful AGI.
High-grade common sense (the sort you’d get by asking any specialist in computer security) says that you should design an AI which you would trust with an open Internet connection, then put it in the box you would use on an untrusted AI during development. (No, the AI will not be angered by this lack of trust and resent you. Thank you for asking.) I think it’s safe to say that for basically everything in FAI strategy (I can’t think of an exception right now) you can identify at least two things supporting any key point, such that either alone was designed to be sufficient independently of the other’s failing, including things like “indirect normativity works” (you try to build in at least some human checks around this which would shut down any scary AI independently of your theory of indirect normativity being remotely correct, while also not trusting the humans to steer the AI because then the humans are your single point of failure).
But didn’t you say that if you find that your FAI design needs to be patched up to resolve some issue, you should rather abandon it and look for another? This redundancy of security looks like a patchup to me. As long as neither the AIs intent nor the boxing is proven to be secure, we are not ready to start development; once one is, we do not need the other.
Redundancy isn’t a design failure or a ‘patch’.
See my interview with Roman here.
Thanks. Pretty depressing, though.