I’d throw in some faithful CoT stuff (e.g. if someone magically completed the agenda I’d been sketching last year at OpenAI, so that we could say “for AIs trained in such-and-such a way, we can trust their CoT to be faithful w.r.t. scheming because they literally don’t have the capability to scheme without getting caught, we tested it; also, these AIs are on a path to AGI; all we have to do is keep scaling them and they’ll get to AGI-except-with-the-faithful-CoT-property.)
I think this is quite likely to happen even ‘by default’ on the current trajectory.
I think this is quite likely to happen even ‘by default’ on the current trajectory.