(A) If we had perfect knowledge of the physical world, or of some part of the physical world, then our uncertainty about X would be completely resolved.
(B) If we had perfect knowledge of some other territory or some part of some other territory (which may itself be imagined!), then our uncertainty about X would be completely resolved.
(C) Some of our uncertainty about X is irreducible, i.e. it cannot be resolved even in principle by observing any territory.
Some claims...
Claim 1: The ordinary case for most realistic Bayesian-ish minds most of the time is to use latent variables which are meaningful and have some predictive utility for the physical world, but cannot be fully resolved even in principle by observing the physical world.
Canonical example: the Boltzman distribution for an ideal gas—not the assorted things people say about the Boltzmann distribution, but the actual math, interpreted as Bayesian probability. The model has one latent variable, the temperature T, and says that all the particle velocities are normally distributed with mean zero and variance proportional to T. Then, just following the ordinary Bayesian math: in order to estimate T from all the particle velocities, I start with some prior P[T], calculate P[T|velocities] using Bayes’ rule, and then for ~any reasonable prior I end up with a posterior distribution over T which is very tightly peaked around the average particle energy… but has nonzero spread. There’s small but nonzero uncertainty in T given all of the particle velocities. And in this simple toy gas model, those particles are the whole world, there’s nothing else to learn about which would further reduce my uncertainty in T.
So at least (C) is a common and realistic case, which is not equivalent to (A), but can still be useful for modeling the physical world.
Claim 2: Any latent can be interpreted as fully resolvable by observing some fictional world, i.e. (C) can always be interpreted as (B).
Think about it for the ideal gas example above: we can view the physical gas particles as a portrayal of the “fictional” temperature T. In the fictional world portrayed, there is an actual physical T from which the particle velocities are generated, and we could just go look at T directly. (You could imagine, for instance, that the “fictional world” is just a python program with a temperature variable and then a bunch of sampling of particle velocities from that variable.) And all the analogies from the post to squirgles or Harry Potter will carry over—including answers to questions like “In what sense is T real?” or “What does it mean for T to change?”.
Here’s how I would operationalize those three:
(A) If we had perfect knowledge of the physical world, or of some part of the physical world, then our uncertainty about X would be completely resolved.
(B) If we had perfect knowledge of some other territory or some part of some other territory (which may itself be imagined!), then our uncertainty about X would be completely resolved.
(C) Some of our uncertainty about X is irreducible, i.e. it cannot be resolved even in principle by observing any territory.
Some claims...
Claim 1: The ordinary case for most realistic Bayesian-ish minds most of the time is to use latent variables which are meaningful and have some predictive utility for the physical world, but cannot be fully resolved even in principle by observing the physical world.
Canonical example: the Boltzman distribution for an ideal gas—not the assorted things people say about the Boltzmann distribution, but the actual math, interpreted as Bayesian probability. The model has one latent variable, the temperature T, and says that all the particle velocities are normally distributed with mean zero and variance proportional to T. Then, just following the ordinary Bayesian math: in order to estimate T from all the particle velocities, I start with some prior P[T], calculate P[T|velocities] using Bayes’ rule, and then for ~any reasonable prior I end up with a posterior distribution over T which is very tightly peaked around the average particle energy… but has nonzero spread. There’s small but nonzero uncertainty in T given all of the particle velocities. And in this simple toy gas model, those particles are the whole world, there’s nothing else to learn about which would further reduce my uncertainty in T.
(See this recent thread for another example involving semantics.)
So at least (C) is a common and realistic case, which is not equivalent to (A), but can still be useful for modeling the physical world.
Claim 2: Any latent can be interpreted as fully resolvable by observing some fictional world, i.e. (C) can always be interpreted as (B).
Think about it for the ideal gas example above: we can view the physical gas particles as a portrayal of the “fictional” temperature T. In the fictional world portrayed, there is an actual physical T from which the particle velocities are generated, and we could just go look at T directly. (You could imagine, for instance, that the “fictional world” is just a python program with a temperature variable and then a bunch of sampling of particle velocities from that variable.) And all the analogies from the post to squirgles or Harry Potter will carry over—including answers to questions like “In what sense is T real?” or “What does it mean for T to change?”.