Keep reading the morality sequence, My comment came while I still had some confusions which are now dissolved.
I don’t know what you count on utility, but I think an AI with your utility function would preserve that which makes you ‘you’ . (it might do anything with your old matter.) At least until it was ready to do something more interesting with ‘you’ .
Pebble sorters value only things that were not pebble sorters, humans value humans, among other things
Keep reading the morality sequence, My comment came while I still had some confusions which are now dissolved.
I don’t know what you count on utility, but I think an AI with your utility function would preserve that which makes you ‘you’ . (it might do anything with your old matter.) At least until it was ready to do something more interesting with ‘you’ .
Pebble sorters value only things that were not pebble sorters, humans value humans, among other things