The danger here is that no small team, even an external one like ARC, no matter how skilled, can elicit the full capabilities or risks of a model the same way the internet inevitably will after deployment. This kind of task necessarily involves extrapolation, and assuming others can go beyond what you can demonstrate. So you need to build thresholds accordingly.
Excellent quote.