Interesting. I was thinking about same regarding the early stages of AGI—it is difficult to define the ‘me’ precisely, and its unclear why would one have a really precise definition of ‘me’ in an early AGI. It’s good enough if the life is ‘me’ to AI but the Jupiter isn’t ‘me’ , that’s a negligible loss in utility from the life not being kosher food for the AI.
Interesting. I was thinking about same regarding the early stages of AGI—it is difficult to define the ‘me’ precisely, and its unclear why would one have a really precise definition of ‘me’ in an early AGI. It’s good enough if the life is ‘me’ to AI but the Jupiter isn’t ‘me’ , that’s a negligible loss in utility from the life not being kosher food for the AI.