I don’t find this convincing. I think the target “dumb enough to be safe, honest, trustworthy, relatively non-agentic, etc., but smart enough to be super helpful for alignment” is narrow (or just nonexistent, using the methods we’re likely to have on hand).
Even if this exists, verification seems extraordinarily difficult: how do we know that the system is being honest? Separately, how do we verify that its solutions are correct? Checking answers is sometimes easier than generating them, but only to a limited degree, and alignment seems like a case where checking is particularly difficult.
It’s also important to keep in mind that on Leopold’s model (and my own), these problems need to be solved under a ton of time pressure. To maintain a lead, the USG in Leopold’s scenario will often need to figure out some of these “under what circumstances can we trust this highly novel system and believe its alignment answers?” issues in a matter of weeks or perhaps months, so that the overall alignment project can complete in a very short window of time. This is not a situation where we’re imagining having a ton of time to develop mastery and deep understanding of these new models. (Or mastery of the alignment problem sufficient to verify when a new idea is on the right track or not.)
It’s also important to keep in mind that on Leopold’s model (and my own), these problems need to be solved under a ton of time pressure. To maintain a lead, the USG in Leopold’s scenario will often need to figure out some of these “under what circumstances can we trust this highly novel system and believe its alignment answers?” issues in a matter of weeks or perhaps months, so that the overall alignment project can complete in a very short window of time. This is not a situation where we’re imagining having a ton of time to develop mastery and deep understanding of these new models. (Or mastery of the alignment problem sufficient to verify when a new idea is on the right track or not.)