What you’re describing above sounds like an aligned AI and I agree that convergence to the best-possible values over time seems like something an aligned AI would do.
But I think you’re mixing up intelligence and values. Sure, maybe an ASI would converge on useful concepts in a way similar to humans. For example, AlphaZero rediscovered some human chess concepts. But because of the orthogonality thesis, intelligence and goals are more or less independent: you can increase the intelligence of a system without its goals changing.
The classic thought experiment illustrating this is Bostrom’s paperclip maximizer which continues to value only paperclips even when it becomes superintelligent.
Also, I don’t think neuromorphic AI would reliably lead to an aligned AI. Maybe an exact whole-brain emulation of some benevolent human would be aligned but otherwise, a neuromorphic AI could have a wide variety of possible goals and most of them wouldn’t be aligned.
What you’re describing above sounds like an aligned AI and I agree that convergence to the best-possible values over time seems like something an aligned AI would do.
But I think you’re mixing up intelligence and values. Sure, maybe an ASI would converge on useful concepts in a way similar to humans. For example, AlphaZero rediscovered some human chess concepts. But because of the orthogonality thesis, intelligence and goals are more or less independent: you can increase the intelligence of a system without its goals changing.
The classic thought experiment illustrating this is Bostrom’s paperclip maximizer which continues to value only paperclips even when it becomes superintelligent.
Also, I don’t think neuromorphic AI would reliably lead to an aligned AI. Maybe an exact whole-brain emulation of some benevolent human would be aligned but otherwise, a neuromorphic AI could have a wide variety of possible goals and most of them wouldn’t be aligned.
I suggest reading The Superintelligent Will to understand these concepts better.
But I did state its goal; to seek out truth (and to utilize anything that might yeild to that effort)