argmaxU, as a mathematical structure, is smarter than god and perfectly aligned to U; the value of argmaxU will never actually be argmaxV because V is more objectively rational, or because you made a typo and it knows you meant to say argmaxV; and no matter how complicated the mapping is from a to U(a) it will never fall short of giving the a that gives the highest value of U.
Which is why in principle you can align a superior being, like argmax, or maybe like a superintelligence.
argmaxU, as a mathematical structure, is smarter than god and perfectly aligned to U; the value of argmaxU will never actually be argmaxV because V is more objectively rational, or because you made a typo and it knows you meant to say argmaxV; and no matter how complicated the mapping is from a to U(a) it will never fall short of giving the a that gives the highest value of U.
Which is why in principle you can align a superior being, like argmax, or maybe like a superintelligence.