“I take it that your goals do not involve avoiding all your distant descendants being systematically obliterated. If you don’t care about such an outcome, fine. I do happen to care.”
In what sense is the descendant (through many iterations of redesign and construction) of an AI solely focused on survival, constructed by some other human my descendant or yours? What features does it possess that make you care about it more than the descendant of an AI constructed by aliens evolved near a distant star? If it’s just a causal relation to your species, then what do you make of the following case: you create your survival machine, and it encounters a similar alien AI, whereupon the two merge, treating the activities of the merged entity as satisfying both of their ‘survival’ aims.
Where does your desire come from? Its achievement wouldn’t advance the preservation of your genes (those would be destroyed), wouldn’t seem to stem from the love of human children, wouldn’t preserve your personality, etc.
“I take it that your goals do not involve avoiding all your distant descendants being systematically obliterated. If you don’t care about such an outcome, fine. I do happen to care.”
In what sense is the descendant (through many iterations of redesign and construction) of an AI solely focused on survival, constructed by some other human my descendant or yours? What features does it possess that make you care about it more than the descendant of an AI constructed by aliens evolved near a distant star? If it’s just a causal relation to your species, then what do you make of the following case: you create your survival machine, and it encounters a similar alien AI, whereupon the two merge, treating the activities of the merged entity as satisfying both of their ‘survival’ aims.
Where does your desire come from? Its achievement wouldn’t advance the preservation of your genes (those would be destroyed), wouldn’t seem to stem from the love of human children, wouldn’t preserve your personality, etc.