Maybe Newcomb’s problem is simply inconceivable to AIXI, in a way that no amount of evidence can ever lead it to expect that the content of the box, and thus the reward, is correlated to its action.
AIXI does recognise this correlation; it two-boxes and with a reasonable amount of evidence it also believes (correctly) that Omega predicted it would two-box.
That’s a possibility, but I find it not very plausible: AIXI world programs contain embeddings of all human minds, and all super-human computable AIs. If we assume that the agent is experienced, world programs embedding these very very smart AIs will get most of probability mass, since they are very good sequence predictors. So if a human can understand Newcomb’s problem, I think that a super-human AI would understand it as well.
The problem is that AIXI cannot recognise the kinds of models in which AIXI’s own action and Omega’s prediction of its action have a common cause (i.e. the AIXI equation). A better agent would be capable of recognising that dependency.
If you always exclude certain kinds of models then it doesn’t matter how smart you are, some explanations are simply never going to occur to you.
If you always exclude certain kinds of models then it doesn’t matter how smart you are, some explanations are simply never going to occur to you.
Actually, these models exist in AIXI world program ensemble. In order to support your point, you have to argue that they are more complex than models which make an incorrect prediction, no matter how much evidence for Newcomb’s problem AIXI has been presented with.
AIXI does recognise this correlation; it two-boxes and with a reasonable amount of evidence it also believes (correctly) that Omega predicted it would two-box.
The problem is that AIXI cannot recognise the kinds of models in which AIXI’s own action and Omega’s prediction of its action have a common cause (i.e. the AIXI equation). A better agent would be capable of recognising that dependency.
If you always exclude certain kinds of models then it doesn’t matter how smart you are, some explanations are simply never going to occur to you.
Actually, these models exist in AIXI world program ensemble. In order to support your point, you have to argue that they are more complex than models which make an incorrect prediction, no matter how much evidence for Newcomb’s problem AIXI has been presented with.