math of AIXI assumes the environment is separably divisible—no matter what you lose, you get a chance to win it back later.
There’s nothing preventing you from running AIXItl in an environment that doesn’t have this property. You lose the optimality results, but if you gave it a careful early training period and let it learn physics before giving it full manipulators and access to its own physical instantiation, it might not kill itself.
You could also build a sense of self into its priors, stating that certain parts of the physical world must be preserved, or else all further future rewards will be zero.
It may be possible to formalize your idea as in Orseau’s “Space-Time Embedded Intelligence,” but it would no longer bear much resemblance to AIXItl. With that said, translating the informal idea you’ve given into math is highly nontrivial. Which parts of its physical world should be preserved and what does that mean in general? AIXI does not even assume our laws of physics.
There’s nothing preventing you from running AIXItl in an environment that doesn’t have this property. You lose the optimality results, but if you gave it a careful early training period and let it learn physics before giving it full manipulators and access to its own physical instantiation, it might not kill itself.
You could also build a sense of self into its priors, stating that certain parts of the physical world must be preserved, or else all further future rewards will be zero.
It may be possible to formalize your idea as in Orseau’s “Space-Time Embedded Intelligence,” but it would no longer bear much resemblance to AIXItl. With that said, translating the informal idea you’ve given into math is highly nontrivial. Which parts of its physical world should be preserved and what does that mean in general? AIXI does not even assume our laws of physics.