Develop a theory of howneural networks function, then apply that theory to either directly align neural networks, or develop an alternative approach to creating AGI that is easier to align. This seems more promising than trying to develop new foundations from scratch, since we already know that neural networks do intelligence-like optimization, the challenge is just figuring out why.
Reverse-engineer neural networks.
Develop a theory of how neural networks function, then apply that theory to either directly align neural networks, or develop an alternative approach to creating AGI that is easier to align. This seems more promising than trying to develop new foundations from scratch, since we already know that neural networks do intelligence-like optimization, the challenge is just figuring out why.