“Humans are trained on how to live on Earth by hours of training on Earth. (...) Maybe most of us are just mimicking how an agent would behave in a given situation.”
I agree that that’s a plausible enough explanation for lots of human behaviour, but I wonder how far you would get in trying to describe historical paradigm shifts using only a ‘mimic hypothesis of agenthood’.
Why would a perfect mimic that was raised on training data of human behaviour do anything paperclip-maximizer-ish? It doesn’t want to mimic being a human, just like Dall-E doesn’t want to generate images, so it doesn’t have a utility function for not wanting to be prevented from mimicking being a human, either.
I agree that that’s a plausible enough explanation for lots of human behaviour, but I wonder how far you would get in trying to describe historical paradigm shifts using only a ‘mimic hypothesis of agenthood’.
Why would a perfect mimic that was raised on training data of human behaviour do anything paperclip-maximizer-ish? It doesn’t want to mimic being a human, just like Dall-E doesn’t want to generate images, so it doesn’t have a utility function for not wanting to be prevented from mimicking being a human, either.