Ok, I’ll concede that AIXI does consider hypotheses in which the environment contains a computable approximation to AIXI and in the near future, the universe will start ignoring AIXI and paying attention to the approximation’s output in the same way it had previously been paying attention to AIXI’s output. Counting that as “identifying itself with the approximation” seems overly generous,
It’s not that AIXI thinks that “the universe will start ignoring AIXI”—the Solomonoff induction part starts by giving weight to an infinite set of models in which AIXI’s actions have no effect whatsoever. It’s that AIXI is learning that there’s this agent running around the universe doing stuff and the universe is responding to it. The identification part happens because the program specifies that the set of bits in the simulated agent’s input registers is the predicted observation stream.
I still don’t see any reason that AIXI would end up considering such a hypothesis likely
Because hypotheses of smaller K-complexity have failed to predict the observation stream. (Not that this is a claim whose truth I’m asserting—just that this is the only reason that Solomonoff induction ever considers an hypothesis likely. I leave open the possibility that a K-simpler universe model that does not defeat Cartesian dualism might exist.)
Acutally, it is weirder than that, because AIXI considers what decisions it will make after its “avatar” is destroyed. Most humans know it doesn’t work that way.
AIXI learns, e.g., that the simulated agent has an actuator, and that all of the effects of the simulated agent’s decisions are mediated through the actuator. It can also predict that if the actuator is destroyed, then the simulated agent’s decisions stop having effects. That’s really all that’s necessary.
It’s not that AIXI thinks that “the universe will start ignoring AIXI”—the Solomonoff induction part starts by giving weight to an infinite set of models in which AIXI’s actions have no effect whatsoever. It’s that AIXI is learning that there’s this agent running around the universe doing stuff and the universe is responding to it. The identification part happens because the program specifies that the set of bits in the simulated agent’s input registers is the predicted observation stream.
Hypotheses in which AIXI’s actions already have no effect on the environment are useless for action guidance; all actions have the same utility.
Because hypotheses of smaller K-complexity have failed to predict the observation stream.
Well yes, I know that is how Solomonoff induction works. But the (useless for action guidance) hypothesis you just suggested is ridiculously high K-complexity, and the hypothesis I suggested has even higher K-complexity. Even worse: these are actually families of hypotheses, parameterized by the the AIXI approximation algorithm being used (and in the case of the hypothesis I suggested, also the time-step on which the switch occurs), and as the number of observations increases, the required accuracy of the AIXI approximation, and thus its K-complexity, also increases. I’m skeptical that this sort of thing could ever end up as a leading hypothesis.
So I have responses, but they’re moot—I found the Cartesian boundary.
Hypotheses in which AIXI’s actions already have no effect on the environment are useless for action guidance; all actions have the same utility.
Fortunately they get falsified and zeroed out right away.
I’m skeptical that this sort of thing could ever end up as a leading hypothesis.
The leading hypothesis has to not get falsified; what you’ve described is the bare minimum required for a Solomonoff inductor to account for an AIXI agent in the environment.
It’s not that AIXI thinks that “the universe will start ignoring AIXI”—the Solomonoff induction part starts by giving weight to an infinite set of models in which AIXI’s actions have no effect whatsoever. It’s that AIXI is learning that there’s this agent running around the universe doing stuff and the universe is responding to it. The identification part happens because the program specifies that the set of bits in the simulated agent’s input registers is the predicted observation stream.
Because hypotheses of smaller K-complexity have failed to predict the observation stream. (Not that this is a claim whose truth I’m asserting—just that this is the only reason that Solomonoff induction ever considers an hypothesis likely. I leave open the possibility that a K-simpler universe model that does not defeat Cartesian dualism might exist.)
AIXI learns, e.g., that the simulated agent has an actuator, and that all of the effects of the simulated agent’s decisions are mediated through the actuator. It can also predict that if the actuator is destroyed, then the simulated agent’s decisions stop having effects. That’s really all that’s necessary.
Hypotheses in which AIXI’s actions already have no effect on the environment are useless for action guidance; all actions have the same utility.
Well yes, I know that is how Solomonoff induction works. But the (useless for action guidance) hypothesis you just suggested is ridiculously high K-complexity, and the hypothesis I suggested has even higher K-complexity. Even worse: these are actually families of hypotheses, parameterized by the the AIXI approximation algorithm being used (and in the case of the hypothesis I suggested, also the time-step on which the switch occurs), and as the number of observations increases, the required accuracy of the AIXI approximation, and thus its K-complexity, also increases. I’m skeptical that this sort of thing could ever end up as a leading hypothesis.
So I have responses, but they’re moot—I found the Cartesian boundary.
Fortunately they get falsified and zeroed out right away.
The leading hypothesis has to not get falsified; what you’ve described is the bare minimum required for a Solomonoff inductor to account for an AIXI agent in the environment.