Yes, I agree with everything you said… until the last sentence ;)
In-parable, your mom is mostly just there as moral support. Neither of you is doing cognitive work the other couldn’t. But an aligned AI might have to do a lot of hard work figuring out what some candidate options for the universe even are, and if we want to check its work it will probably have to break big complicated visions of the future into human-comprehensible mouthfuls. So we really will need to extend trust to it—not as in trusting that whatever it picks is the only right thing, but trusting it to make decent decisions even in domains that are too complicated for human oversight.
Yes, I agree with everything you said… until the last sentence ;)
In-parable, your mom is mostly just there as moral support. Neither of you is doing cognitive work the other couldn’t. But an aligned AI might have to do a lot of hard work figuring out what some candidate options for the universe even are, and if we want to check its work it will probably have to break big complicated visions of the future into human-comprehensible mouthfuls. So we really will need to extend trust to it—not as in trusting that whatever it picks is the only right thing, but trusting it to make decent decisions even in domains that are too complicated for human oversight.