I commented on the previous post a few days after it went up detailing some misgivings about the arguments presented there (I guess you missed my comment). I was reading this post with burgeoning hope that my misgivings would be inadvertently addressed, and then I encountered this:
AIXI doesn’t know that its future behaviors depend on a changeable, material object implementing its memories. The notion isn’t even in its hypothesis space.
But if “naturalized induction” is a computer program, then the notion is in AIXI’s hypothesis space—by definition.
I saw your comment; the last section (‘Beyond Solomonoff?‘) speaks to the worry you raised. Somewhere in AIXI’s hypothesis space is a reasoner R that is a reductionist about R; AIXI can simulate human scientists, for example. But nowhere in AIXI’s hypothesis space is a reasoner that is a native representation of AIXI as ‘me’, as the agent doing the hypothesizing.
One way I’d put this is that AIXI can entertain every physical hypothesis, but not every indexical hypothesis. Being able to consider all the objective ways the world could be doesn’t mean you’re able to consider all the ways you could be located in the world.
AIXI’s hypothesis space does include experts on AIXI that could give it advice about how best to behave like a naturalist. Here the problem isn’t that the hypotheses are missing, but that they don’t look like they’ll be assigned a reasonable prior probability.
But nowhere in AIXI’s hypothesis space is a reasoner that is a native representation of AIXI as ‘me’, as the agent doing the hypothesizing.
I disagree: Among all the world-programs in AIXI model space, there are some programs where, after AIXI performs one action, all its future actions are ignored and control is passed to a subroutine “AGENT” in the program. In principle AIXI can reason that if the last action it performs damages AGENT, e.g. by dropping an anvil on its head, the reward signal, computed by some reward subroutine in the world-program, won’t be maximized anymore.
Of course there are the usual computability issues: the true AIXI is uncomputable, hence the AGENTs would be actually a complexity-weighted mixture of its computable approximations. AIXItl would have the same issue w.r.t. the resource bounds t and l. I’m not sure this is necessarily a severe issue. Anyway, I suppose that AIXItl could be modified in some UDT-like way to include a quined source code and recognize copies of itself inside the world-programs.
The other issue is how does AIXI learn to assign high weights to these world-programs in a non-ergodic environment? Humans seem to manage to do that by a combination of innate priors and tutoring. I suppose that something similar is in principle applicable to AIXI.
It seems worth saying at this point that I don’t have an objection to loading up an AI with true prior information; it’s just not clear to me that a Solomonoff approximator would be incapable of learning that it’s part of the Universe and that its continued existence is contingent on the persistence of some specific structure in the Universe.
But, just as HALT-predicting programs are more complex than immortalist programs, other RADICAL-TRANSFORMATION-OF-EXPERIENCE-predicting programs are too. For every program in AIXI’s ensemble that’s a reductionist, there will be simpler agents that mimic the reductionist’s retrodictions and then make non-naturalistic predictions.
So this seems to be the root of the problem. Contrary to what you argued in the previous post, my intuition is that the programs that make non-naturalistic predictions are not shorter. Generically non-naturalistic programs get ruled out during the process of learning how the world works; programs that make non-naturalistic predictions specifically about what AIXI(tl) will experience after smashing itself have to treat the chunk of the Universe carrying out the computation as special, which is what makes them less simple than programs that do not single out that chunk of the Universe as special.
As you can see, my intuition is quite at odds with the intuition inspired by noticing that programs with a HALT instruction are always longer than programs that just chop off said HALT instruction.
programs that make non-naturalistic predictions specifically about what AIXI(tl) will experience after smashing itself have to treat the chunk of the Universe carrying out the computation as special,
Well, any program AIXI gives weight must regard that chunk of the universe as special. After all, it is that chunk that correlates with AIXI’s inputs and actions, and indeed the only reason this universe is considered as a hypothesis is so that that chunk would have those correlations.
The kind of “special” you’re talking about is learnable (and in accord with naturalistic predictions); the kind of “special” I’m talking about is false#Arguments_against_dualism).
I commented on the previous post a few days after it went up detailing some misgivings about the arguments presented there (I guess you missed my comment). I was reading this post with burgeoning hope that my misgivings would be inadvertently addressed, and then I encountered this:
But if “naturalized induction” is a computer program, then the notion is in AIXI’s hypothesis space—by definition.
Going back to the post to read some more...
I saw your comment; the last section (‘Beyond Solomonoff?‘) speaks to the worry you raised. Somewhere in AIXI’s hypothesis space is a reasoner R that is a reductionist about R; AIXI can simulate human scientists, for example. But nowhere in AIXI’s hypothesis space is a reasoner that is a native representation of AIXI as ‘me’, as the agent doing the hypothesizing.
One way I’d put this is that AIXI can entertain every physical hypothesis, but not every indexical hypothesis. Being able to consider all the objective ways the world could be doesn’t mean you’re able to consider all the ways you could be located in the world.
AIXI’s hypothesis space does include experts on AIXI that could give it advice about how best to behave like a naturalist. Here the problem isn’t that the hypotheses are missing, but that they don’t look like they’ll be assigned a reasonable prior probability.
I disagree: Among all the world-programs in AIXI model space, there are some programs where, after AIXI performs one action, all its future actions are ignored and control is passed to a subroutine “AGENT” in the program. In principle AIXI can reason that if the last action it performs damages AGENT, e.g. by dropping an anvil on its head, the reward signal, computed by some reward subroutine in the world-program, won’t be maximized anymore.
Of course there are the usual computability issues: the true AIXI is uncomputable, hence the AGENTs would be actually a complexity-weighted mixture of its computable approximations. AIXItl would have the same issue w.r.t. the resource bounds t and l.
I’m not sure this is necessarily a severe issue. Anyway, I suppose that AIXItl could be modified in some UDT-like way to include a quined source code and recognize copies of itself inside the world-programs.
The other issue is how does AIXI learn to assign high weights to these world-programs in a non-ergodic environment? Humans seem to manage to do that by a combination of innate priors and tutoring. I suppose that something similar is in principle applicable to AIXI.
It seems worth saying at this point that I don’t have an objection to loading up an AI with true prior information; it’s just not clear to me that a Solomonoff approximator would be incapable of learning that it’s part of the Universe and that its continued existence is contingent on the persistence of some specific structure in the Universe.
So this seems to be the root of the problem. Contrary to what you argued in the previous post, my intuition is that the programs that make non-naturalistic predictions are not shorter. Generically non-naturalistic programs get ruled out during the process of learning how the world works; programs that make non-naturalistic predictions specifically about what AIXI(tl) will experience after smashing itself have to treat the chunk of the Universe carrying out the computation as special, which is what makes them less simple than programs that do not single out that chunk of the Universe as special.
As you can see, my intuition is quite at odds with the intuition inspired by noticing that programs with a HALT instruction are always longer than programs that just chop off said HALT instruction.
Well, any program AIXI gives weight must regard that chunk of the universe as special. After all, it is that chunk that correlates with AIXI’s inputs and actions, and indeed the only reason this universe is considered as a hypothesis is so that that chunk would have those correlations.
The kind of “special” you’re talking about is learnable (and in accord with naturalistic predictions); the kind of “special” I’m talking about is false#Arguments_against_dualism).