CFAR’s working documents and notes could help a lot, in a specific scenario.
If most of the training that an emerging AGI does is with the history of human rationality, that could yield some really valuable research. If heavy weight is placed on the successes, failures, paths that were touched on but then dropped, etc, in addition to the polished publications, a halfway-finished AGI would be in the best possible position to combine that information with its half-AGI capabilities and all its other training data (potentially including lots of fMRI data of people trying to be rational) and pump out some extremely strong techniques for creating powerful thinkers (at that point, of course, it would be paused for as long as possible in the hopes that one of the augmented people finds a solution in time).
Unfortunately, it would still be finishing the job during crunch time, which is much later than ideal. But it would still finish the job, and there would definitely end up being people on earth who are really really good at thinking of a solution for alignment.
CFAR’s working documents and notes could help a lot, in a specific scenario.
If most of the training that an emerging AGI does is with the history of human rationality, that could yield some really valuable research. If heavy weight is placed on the successes, failures, paths that were touched on but then dropped, etc, in addition to the polished publications, a halfway-finished AGI would be in the best possible position to combine that information with its half-AGI capabilities and all its other training data (potentially including lots of fMRI data of people trying to be rational) and pump out some extremely strong techniques for creating powerful thinkers (at that point, of course, it would be paused for as long as possible in the hopes that one of the augmented people finds a solution in time).
Unfortunately, it would still be finishing the job during crunch time, which is much later than ideal. But it would still finish the job, and there would definitely end up being people on earth who are really really good at thinking of a solution for alignment.