First, AIXI’s complexity penalty does not treat actions in the same way as percepts, they are basically “free” because of how Professor Hutter extended Solomonoff induction to chronological environments. So this whole discussion seems to apply mainly to a) cases in which AIXI’s actions can be corrupted, discussed later or b) one uses an alternative definition of AIXI “directly” in terms of Solomonoff induction, which is essentially an embedded version. I am interested in this extension, have implemented an approximation that seems to work okay empirically, and I am not aware of any exact appearances in the literature. MIRI’s reflective version of AIXI is probably the closest.
Second, the argument for “instability” is weak. As far as I can tell, as long as AIXI (in either formulation) has complete control of its actions it would just ignore the possibility that the environment is computing them because this should cancel out of expected utility calculations.
I think parts of this are wrong.
First, AIXI’s complexity penalty does not treat actions in the same way as percepts, they are basically “free” because of how Professor Hutter extended Solomonoff induction to chronological environments. So this whole discussion seems to apply mainly to a) cases in which AIXI’s actions can be corrupted, discussed later or b) one uses an alternative definition of AIXI “directly” in terms of Solomonoff induction, which is essentially an embedded version. I am interested in this extension, have implemented an approximation that seems to work okay empirically, and I am not aware of any exact appearances in the literature. MIRI’s reflective version of AIXI is probably the closest.
Second, the argument for “instability” is weak. As far as I can tell, as long as AIXI (in either formulation) has complete control of its actions it would just ignore the possibility that the environment is computing them because this should cancel out of expected utility calculations.
I have carried this argument somewhat further and argued that a modified off-policy version of AIXI can be taught to avoid the anvil problem here: https://www.lesswrong.com/posts/WECqiLtQiisqWvhim/free-will-and-dodging-anvils-aixi-off-policy