Relative difficulty of Technical Alignment work and power-acquisition for an AI: Given a self-improving AI with some take-off speed, does it become able to solve the technical alignment problem before it becomes able to escape whatever confinement is in place to stop it?
A team with safety concerns in possession of an AI in the critical periods might wish to use it to solve the alignment problem, if solving the problem requires a dumber AI than one that is capable of escaping confinement, then the team can try to turn-off the self-improvement loop at the level required for alignment work and bootstrap a safe-AI from there.
Relative difficulty of Technical Alignment work and power-acquisition for an AI: Given a self-improving AI with some take-off speed, does it become able to solve the technical alignment problem before it becomes able to escape whatever confinement is in place to stop it?
A team with safety concerns in possession of an AI in the critical periods might wish to use it to solve the alignment problem, if solving the problem requires a dumber AI than one that is capable of escaping confinement, then the team can try to turn-off the self-improvement loop at the level required for alignment work and bootstrap a safe-AI from there.