Sadly, the difference in their priors could still make a big difference for the natural latents, due to the tiny mixtures problem.
Currently our best way to handle this is to assume a universal prior. That still allows for a wide variety of different priors (i.e. any Turing machine), but the Solomonoff version of natural latents doesn’t have the tiny mixtures problem. For Solomonoff natural latents, we do have the sort of result you’re intuiting, where the divergence (in bits) between the two agents’ priors just gets added to the error term on all the approximations.
Sadly, the difference in their priors could still make a big difference for the natural latents, due to the tiny mixtures problem.
Currently our best way to handle this is to assume a universal prior. That still allows for a wide variety of different priors (i.e. any Turing machine), but the Solomonoff version of natural latents doesn’t have the tiny mixtures problem. For Solomonoff natural latents, we do have the sort of result you’re intuiting, where the divergence (in bits) between the two agents’ priors just gets added to the error term on all the approximations.