The major difference between gene-brain and human-AI is that there is an evolutionary feedback loop between genes and the brains they produce.
It is not clear that an AI, which kills its ‘brains’ will be less fit.
Maybe an AI, which goes from mostly serving to mostly parasitizing its brain predecessors, can still provide enough value to humans while largely doing its own, incomprehensible superintelligent stuff.
Whatever fitness will mean for these AIs though, they will play out their own evolutionary games within their world where compute, network bandwidth and storage are the amino acids and sunshine.
Darwinian forces shall govern those AIs development, 100% oblivious to us, as we will be orders of magnitude too slow and stupid to understand them, by definition.
The major difference between gene-brain and human-AI is that there is an evolutionary feedback loop between genes and the brains they produce.
It is not clear that an AI, which kills its ‘brains’ will be less fit.
Maybe an AI, which goes from mostly serving to mostly parasitizing its brain predecessors, can still provide enough value to humans while largely doing its own, incomprehensible superintelligent stuff.
Whatever fitness will mean for these AIs though, they will play out their own evolutionary games within their world where compute, network bandwidth and storage are the amino acids and sunshine.
Darwinian forces shall govern those AIs development, 100% oblivious to us, as we will be orders of magnitude too slow and stupid to understand them, by definition.