I have never had much patience for Hanson and it seems someone as intelligent as himself should know that values emerge from circumstance. What use, for example, would AI have for romantic love in a world where procreation consists of digital copies? What use are coordinated behaviors for society if lies are impossible and you can just populate your “society” with clones of yourself? What use is there for taste without the evolutionary setup for sugars, etc.
Behaviors arise from environmental conditions, and its just wild to see a thought that eliminating all of that would give us anything similar.
Essentially the only value you will preserve is the universal one for power seeking. I like to think very few of us want to value power seeking over love and cooperation: right now, Hanson is valuing the “power” of his “descendants” over their ability to be human, why would AI be different?
I also believe that animal life and cognition has value, as their own form of non-human intelligence. An AI catastrope that eliminates the biosphere seems to be vastly negative, immoral and agency-reducing for them: they didn’t vote to go extinct.
I have never had much patience for Hanson and it seems someone as intelligent as himself should know that values emerge from circumstance. What use, for example, would AI have for romantic love in a world where procreation consists of digital copies? What use are coordinated behaviors for society if lies are impossible and you can just populate your “society” with clones of yourself? What use is there for taste without the evolutionary setup for sugars, etc.
Behaviors arise from environmental conditions, and its just wild to see a thought that eliminating all of that would give us anything similar.
Essentially the only value you will preserve is the universal one for power seeking. I like to think very few of us want to value power seeking over love and cooperation: right now, Hanson is valuing the “power” of his “descendants” over their ability to be human, why would AI be different?
I also believe that animal life and cognition has value, as their own form of non-human intelligence. An AI catastrope that eliminates the biosphere seems to be vastly negative, immoral and agency-reducing for them: they didn’t vote to go extinct.