I personally don’t see the choice of “allowing a more intelligent set of agents take over” as particularly altruistic: personally, i think intelligence trumps species, and I am not convinced interrupting its growth to make sure more sets of genes similar to mine find hosts for longer would somehow be “for my benefit”.
Even in my AI Risk years, what I was afraid is the same I’m afraid of now: Boring Futures. The difference is that in the meantime the arguments for a singleton ASI, with a single unchangeable utility function that is not more intelligence/knowledge/curiosity became less and less tenable (together with FOOM within our lifetimes).
This being the case, “altruistic” really seems out of place: it’s likely that early sapiens would have understood nothing of our goals, our morality, and the drives that got us to build civilisations—but would it have been better for them had they murdered the first guy in the troop they found flirting with a neanderthal and prevented this? I personally doubt it, and I think the comparison between us and ASI is more or less in the same ballpark,
I personally don’t see the choice of “allowing a more intelligent set of agents take over” as particularly altruistic: personally, i think intelligence trumps species, and I am not convinced interrupting its growth to make sure more sets of genes similar to mine find hosts for longer would somehow be “for my benefit”.
Even in my AI Risk years, what I was afraid is the same I’m afraid of now: Boring Futures. The difference is that in the meantime the arguments for a singleton ASI, with a single unchangeable utility function that is not more intelligence/knowledge/curiosity became less and less tenable (together with FOOM within our lifetimes).
This being the case, “altruistic” really seems out of place: it’s likely that early sapiens would have understood nothing of our goals, our morality, and the drives that got us to build civilisations—but would it have been better for them had they murdered the first guy in the troop they found flirting with a neanderthal and prevented this? I personally doubt it, and I think the comparison between us and ASI is more or less in the same ballpark,