Then the first step is that A asks what happens if its next output is (say) 0. To do that it needs to run H to produce the next bit of output. But running H involves running a simulation of A, and inside that simulation the exact same situation arises, namely that sim(A) considers various outputs that it might make and runs simulations of the world, resulting in another level of recursion to sim(sim(A)), and so on in an infinite loop.
This seems to be the observation that you can’t have a Turing machine that implements AIXI. An approximate AIXI is not going to be able to simulate itself.
Is it possible to make Model 2 just slightly simpler by somehow leveraging the “free” information on the output tape?
I don’t think this is possible, although it is an interesting thought. The main issue is that before you get to leverage the first N bits of AIXI’s output you have to also explain the first N bits of AIXI’s input, which seems basically guaranteed to wash out the complexity gains (because all of the info in the first N bits of AIXI’s output was coming from the first N bits of AIXI’s input).
This seems to be the observation that you can’t have a Turing machine that implements AIXI. An approximate AIXI is not going to be able to simulate itself.
Yes, I guess you’re right. But doesn’t this also mean that no computable approximation of AIXI will ever hypothesize a world that contains a model of itself, for if it did then it will go into the infinite loop I described. So it seems the problem of Model 2 will never come up?
The main issue is that before you get to leverage the first N bits of AIXI’s output you have to also explain the first N bits of AIXI’s input
Not sure I’m understanding you correctly but this seems wrong. AIXI conditions on all its outputs so far, right? So if the world is a bit-repeater then one valid model of the world is literally a bit repeater, which explains the inputs but not the outputs.
This seems to be the observation that you can’t have a Turing machine that implements AIXI. An approximate AIXI is not going to be able to simulate itself.
I don’t think this is possible, although it is an interesting thought. The main issue is that before you get to leverage the first N bits of AIXI’s output you have to also explain the first N bits of AIXI’s input, which seems basically guaranteed to wash out the complexity gains (because all of the info in the first N bits of AIXI’s output was coming from the first N bits of AIXI’s input).
Yes, I guess you’re right. But doesn’t this also mean that no computable approximation of AIXI will ever hypothesize a world that contains a model of itself, for if it did then it will go into the infinite loop I described. So it seems the problem of Model 2 will never come up?
Not sure I’m understanding you correctly but this seems wrong. AIXI conditions on all its outputs so far, right? So if the world is a bit-repeater then one valid model of the world is literally a bit repeater, which explains the inputs but not the outputs.