So although Model 2 may make up most of the probability mass, it contributes nothing important to the expected value computation determining A’s actions.
Given this fact, I couldn’t follow the discussion of A’s tendency to believe Model 2 in “approximate AIXI” section. Model 2 doesn’t have any effect, so why worry about its probability? What matters is relative probability of models that do depend on their arguments.
If Model 2 is more likely than Model 1, behavior is basically guaranteed to be arbitrary.
How so? The part of the distribution that matters consists of the programs that respond to input, so we might as well stipulate that we only allow such programs in the distribution, and “Model 2” can’t be part of it. This restriction doesn’t dramatically change the original form of the decision algorithm.
Given this fact, I couldn’t follow the discussion of A’s tendency to believe Model 2 in “approximate AIXI” section. Model 2 doesn’t have any effect, so why worry about its probability? What matters is relative probability of models that do depend on their arguments.
There are only slightly more complicated models (e.g. Model 3) which are indistinguishable from model 2 but do depend on their arguments.
If Model 2 is more likely than Model 1, behavior is basically guaranteed to be arbitrary.
How so? The part of the distribution that matters consists of the programs that respond to input, so we might as well stipulate that we only allow such programs in the distribution, and “Model 2” can’t be part of it. This restriction doesn’t dramatically change the original form of the decision algorithm.