I also didn’t initially buy the argument that Marcus gave and I think some modifications and care are required to make AIXI work as an embedded agent—the off-policy version is a start. Still, I think there are reasonable responses to the objections you have made:
1: It would be standard to issue a negative reward (or decrease the positive reward) if AIXI is at risk of harming its body. This is the equivalent. 2: AIXI does not believe in heaven. If its percept stream ends this is treated as 0 reward forever (which is usually but not always taken as the worst reward possible depending on author). It’s unclear if AIXI would expect the destruction of its body to lead to the end of its percept stream, but I think it would under some conditions.
It could be difficult to explain to AIXI what “its body” is.
I think the entire point of AIXI was that it kinda considers all possible universes with all possible laws of physics, and then updates based on evidence. To specify “its body”, you would need to explain many things about our universe, which I think defeats the purpose of having AIXI.
Any time you attempt to implement AIXI (or any approximation) in the real world you must specify the reward mechanism. If AIXI is equipped with a robotic body you could choose for the sensors to provide “pain” signals. There is no need to provide a nebulous definition of what is or is not part of AIXI’s body in order to achieve this.
Ah, that makes sense! AIXI can receive pain signals long before it knows what they “mean”, and as its model of the world improves, it learns to avoid pain.
I also didn’t initially buy the argument that Marcus gave and I think some modifications and care are required to make AIXI work as an embedded agent—the off-policy version is a start. Still, I think there are reasonable responses to the objections you have made:
1: It would be standard to issue a negative reward (or decrease the positive reward) if AIXI is at risk of harming its body. This is the equivalent.
2: AIXI does not believe in heaven. If its percept stream ends this is treated as 0 reward forever (which is usually but not always taken as the worst reward possible depending on author). It’s unclear if AIXI would expect the destruction of its body to lead to the end of its percept stream, but I think it would under some conditions.
It could be difficult to explain to AIXI what “its body” is.
I think the entire point of AIXI was that it kinda considers all possible universes with all possible laws of physics, and then updates based on evidence. To specify “its body”, you would need to explain many things about our universe, which I think defeats the purpose of having AIXI.
Any time you attempt to implement AIXI (or any approximation) in the real world you must specify the reward mechanism. If AIXI is equipped with a robotic body you could choose for the sensors to provide “pain” signals. There is no need to provide a nebulous definition of what is or is not part of AIXI’s body in order to achieve this.
Ah, that makes sense! AIXI can receive pain signals long before it knows what they “mean”, and as its model of the world improves, it learns to avoid pain.