The issue is that besides the computable beings S.I. includes a lot of crud that is predicting past observations without containing a ‘compiler’ for some concept of ‘hey, the universe might be doing something uncomputable here, we better come up with an intricate experiment to test this because having a hypercomputer will be awesome’. The shorter crud influences actions the most. edit: also it is unclear how well can AIXI seek experimentation. We do experiments because we guess that with more knowledge we’ll have more power. It’s like self improvement, it requires some reflective thinking. For AIXI its like seeking futures where the predictor works bad.
Easier example: We can make a theory containing true real numbers, without penalizing theory for the code for discretization and for construction of a code that recreates past observations. We can assign some vague ‘simplicity’ values to theories that are not computable, namely a theory with true reals is simpler than discretization of this theory with a made-up discretization constant. We don’t rate the theories by size of their implementation by applied mathematicians. S.I. does rate by size of discrete, computable implementation. It’s no help if S.I. will contain somewhere neatly separable compiler plus our theory, if the weight for this is small.
The issue is that besides the computable beings S.I. includes a lot of crud that is predicting past observations without containing a ‘compiler’ for some concept of ‘hey, the universe might be doing something uncomputable here, we better come up with an intricate experiment to test this because having a hypercomputer will be awesome’. The shorter crud influences actions the most. edit: also it is unclear how well can AIXI seek experimentation. We do experiments because we guess that with more knowledge we’ll have more power. It’s like self improvement, it requires some reflective thinking. For AIXI its like seeking futures where the predictor works bad.
Easier example: We can make a theory containing true real numbers, without penalizing theory for the code for discretization and for construction of a code that recreates past observations. We can assign some vague ‘simplicity’ values to theories that are not computable, namely a theory with true reals is simpler than discretization of this theory with a made-up discretization constant. We don’t rate the theories by size of their implementation by applied mathematicians. S.I. does rate by size of discrete, computable implementation. It’s no help if S.I. will contain somewhere neatly separable compiler plus our theory, if the weight for this is small.