There’s a whole lot to respond to here, and it may take the length of Surfing Uncertainty to do so. I’ll point instead to one key dimension.
You’re discussing PP as a possible model for AI, whereas I posit PP as a model for animal brains. The main difference is that animal brains are evolved and occur inside bodies.
Evolution is the answer to the dark room problem. You come with prebuilt hardware that is adapted a certain adaptive niche, which is equivalent to modeling it. Your legs are a model of the shape of the ground and the size of your evolutionary territory. Your color vision is a model of berries in a bush, and your fingers that pick them. Your evolved body is a hyperprior you can’t update away. In a sense, you’re predicting all the things that are adaptive: being full of good food, in the company of allies and mates, being vigorous and healthy, learning new things. Lying hungry in a dark room creates a persistent error in your highest-order predictive models (the evolved ones) that you can’t change.
Your evolved prior supposes that you have a body, and that the way you persist over time is by using that body. You are not a disembodied agent learning things for fun or getting scored on some limited test of prediction or matching. Everything your brain does is oriented towards acting on the world effectively.
You can see that perception and action rely on the same mechanism in many ways, starting with the simple fact that when you look at something you don’t receive a static picture, but rather constantly saccade and shift your eyes, contract and expand your pupil and cornea, move your head around, and also automatically compensate for all of this motion. None of this is relevant to an AI who processes images fed to it “out of the void”, and whose main objective function is something other than maintaining homeostasis of a living, moving body.
Zooming out, Friston’s core idea is a direct consequence of thermodynamics: for any system (like an organism) to persist in a state of low entropy (e.g. 98°F) in an environment that is higher entropy but contains some exploitable order (e.g. calories aren’t uniformly spread in the universe but concentrated in bananas), it must exploit this order. Exploiting it is equivalent to minimizing surprise, since if you’re surprised there some pattern of the world that you failed to make use of (free energy).
Now if you just apply this basic principle to your genes persisting over an evolutionary time scale and your body persisting over the time scale of decades and this sets the stage for PP applied to animals.
Zooming out, Friston’s core idea is a direct consequence of thermodynamics: for any system (like an organism) to persist in a state of low entropy (e.g. 98°F) in an environment that is higher entropy but contains some exploitable order (e.g. calories aren’t uniformly spread in the universe but concentrated in bananas), it must exploit this order. Exploiting it is equivalent to minimizing surprise, since if you’re surprised there some pattern of the world that you failed to make use of (free energy).
I haven’t yet understood the mathematical details of Friston’s arguments. I’ve been told that some of them are flawed. But it’s plausible to me that the particular mathematical argument you’re pointing at here is OK. However, I doubt the conclusion of that argument would especially convince me that the brain is set up with the particular sort of architecture described by PP. This, it seems to me, gets into the domain of PP as a theoretical model of ideal agency as opposed to a specific neurological hypothesis.
Humans did not perfectly inherit the abstract goals which would have been most evolutionary beneficial. We are not fitness-maximizers. Similarly, even if all intelligent beings need to avoid entropy in order to keep living, that does not establish that we are entropy-minimizers at the core of our motivation system. As per my sibling comment, that’s like looking at a market economy and concluding that everyone is a money-maximizer. It’s not a necessary supposition, because we can also explain everyone’s money-seeking behavior by pointing out that money is very useful.
You can see that perception and action rely on the same mechanism in many ways, starting with the simple fact that when you look at something you don’t receive a static picture, but rather constantly saccade and shift your eyes, contract and expand your pupil and cornea, move your head around, and also automatically compensate for all of this motion.
How does this suggest that perception and action rely on the same mechanism, as opposed to are very intertwined? I would certainly agree that motor control in vision has tight feedback loops with vision itself. What I don’t believe is that we should model this as acting so as to minimize prediction loss. For one thing, I’ve read that a pretty good model of saccade movement patterns is that we look at the most surprising parts of the image, which would be better-modeled by moving eyes so as to maximize predictive loss.
Babies look longer at objects which they find surprising, as opposed to those which they recognize.
It’s true that PP can predict some behaviors like this, because you’d do this in order to learn, so that you minimize future prediction error. But that doesn’t mean PP is helping us predict those eye movements.
In a world dependent on money, a money-minimizing person might still have to obtain and use money in order to survive and get to a point where they can successfully do without money. That doesn’t mean we can look at money-seeking behavior and conclude that a person is a money-minimizer. More likely that they’re a money-maximizer. But they could be any number of things, because in this world, you have to deal with money in a broad variety of circumstances.
Let me briefly sketch an anti-PP theory. According to what you’ve said so far, I understand you as saying that we act in a way which minimizes prediction error, but according to a warped prior which doesn’t just try to model reality statistically accurately, but rather, increases the probability of things like food, sex, etc in accordance with their importance (to evolutionary fitness). This causes us to seek those things.
My anti-PP theory is this: we act in a way which maximizes prediction error, but according to a warped prior which doesn’t just model reality statistically accurately, but rather, decreases the probability of things like food, sex, etc in accordance with their importance. This causes us to seek those things.
I don’t particularly believe anti-PP, but I find it to be more plausible than PP. It fits human behavior better. It fits eye saccades better. (The eye hits surprising parts of the image, plus sexually significant parts of the image. It stands to reason that sexually significant images are artificially “surprising” to our visual system, making them more interesting.) It fits curiosity and play behavior better.
By the way, I’m actually much more amenable to the version of PP in Kaj Sotala’s post on craving, where warping epistemics by forcing belief in success is just one motivation among several in the brain. I do think something similar to that seems to happen, although my explanation for it is much different (see my earlier comment). I just don’t buy that this is the basic action mechanism of the brain, governing all our behavior, since it seems like a large swath of our behavior is basically the opposite of what you’d expect under this hypothesis. Yes, these predictions can always be fixed by sufficiently modifying the prior, forcing the “pursuing minimal prediction error” hypothesis to line up with the data we see. However, because humans are curious creatures who look at surprising things, engage in experimental play, and like to explore, you’re going to have to take a sensible probability distribution and just about reverse the probabilities to explain those observations. At that point, you might as well switch to anti-PP theory.
You’re discussing PP as a possible model for AI, whereas I posit PP as a model for animal brains. The main difference is that animal brains are evolved and occur inside bodies.
So, for your project of re-writing rationality in PP, would PP constitute a model of human irrationality, and how to rectify it, in contrast to ideal rationality (which would not be well-described by PP)?
Or would you employ PP both as a model which explains human irrationality and as an ideal rationality notion, so that we can use it both as the framework in which we describe irrationality and as the framework in which we can understand what better rationality would be?
Evolution is the answer to the dark room problem. You come with prebuilt hardware that is adapted a certain adaptive niche, which is equivalent to modeling it. Your legs are a model of the shape of the ground and the size of your evolutionary territory. Your color vision is a model of berries in a bush, and your fingers that pick them. Your evolved body is a hyperprior you can’t update away. In a sense, you’re predicting all the things that are adaptive: being full of good food, in the company of allies and mates, being vigorous and healthy, learning new things. Lying hungry in a dark room creates a persistent error in your highest-order predictive models (the evolved ones) that you can’t change.
Am I right in inferring from this that your preferred version of PP is one where we explicitly plan to minimize prediction error, as opposed to the Active Inference model (which instead minimizes KL divergence)? Or do you endorse an Active Inference type model?
This explanation in terms of evolution makes the PP theory consistent with observations, but does not give me a reason to believe PP. The added complexity to the prior is similar to the added complexity of other kinds of machinery to implement drives, so as yet I see no reason to prefer this explanation to other possibly explanations of what’s going on in the brain.
My remarks about problems with different versions of PP can each be patched in various ways; these are not supposed to be “gotcha” arguments in the sense of “PP can’t explain this! / PP can’t deal with this!”. Rather, I’m trying to boggle at why PP looks promising in the first place, as a hypothesis to raise to our attention.
Each of the arguments I mentioned are about one way I might see that someone might think PP is doing some work for us, and why I don’t see that as a promising avenue.
So I remain curious what the generators of your view are.
There’s a whole lot to respond to here, and it may take the length of Surfing Uncertainty to do so. I’ll point instead to one key dimension.
You’re discussing PP as a possible model for AI, whereas I posit PP as a model for animal brains. The main difference is that animal brains are evolved and occur inside bodies.
Evolution is the answer to the dark room problem. You come with prebuilt hardware that is adapted a certain adaptive niche, which is equivalent to modeling it. Your legs are a model of the shape of the ground and the size of your evolutionary territory. Your color vision is a model of berries in a bush, and your fingers that pick them. Your evolved body is a hyperprior you can’t update away. In a sense, you’re predicting all the things that are adaptive: being full of good food, in the company of allies and mates, being vigorous and healthy, learning new things. Lying hungry in a dark room creates a persistent error in your highest-order predictive models (the evolved ones) that you can’t change.
Your evolved prior supposes that you have a body, and that the way you persist over time is by using that body. You are not a disembodied agent learning things for fun or getting scored on some limited test of prediction or matching. Everything your brain does is oriented towards acting on the world effectively.
You can see that perception and action rely on the same mechanism in many ways, starting with the simple fact that when you look at something you don’t receive a static picture, but rather constantly saccade and shift your eyes, contract and expand your pupil and cornea, move your head around, and also automatically compensate for all of this motion. None of this is relevant to an AI who processes images fed to it “out of the void”, and whose main objective function is something other than maintaining homeostasis of a living, moving body.
Zooming out, Friston’s core idea is a direct consequence of thermodynamics: for any system (like an organism) to persist in a state of low entropy (e.g. 98°F) in an environment that is higher entropy but contains some exploitable order (e.g. calories aren’t uniformly spread in the universe but concentrated in bananas), it must exploit this order. Exploiting it is equivalent to minimizing surprise, since if you’re surprised there some pattern of the world that you failed to make use of (free energy).
Now if you just apply this basic principle to your genes persisting over an evolutionary time scale and your body persisting over the time scale of decades and this sets the stage for PP applied to animals.
For more, here’s a conversation between Clark, Friston, and an information theorist about the Dark Room problem.
I haven’t yet understood the mathematical details of Friston’s arguments. I’ve been told that some of them are flawed. But it’s plausible to me that the particular mathematical argument you’re pointing at here is OK. However, I doubt the conclusion of that argument would especially convince me that the brain is set up with the particular sort of architecture described by PP. This, it seems to me, gets into the domain of PP as a theoretical model of ideal agency as opposed to a specific neurological hypothesis.
Humans did not perfectly inherit the abstract goals which would have been most evolutionary beneficial. We are not fitness-maximizers. Similarly, even if all intelligent beings need to avoid entropy in order to keep living, that does not establish that we are entropy-minimizers at the core of our motivation system. As per my sibling comment, that’s like looking at a market economy and concluding that everyone is a money-maximizer. It’s not a necessary supposition, because we can also explain everyone’s money-seeking behavior by pointing out that money is very useful.
How does this suggest that perception and action rely on the same mechanism, as opposed to are very intertwined? I would certainly agree that motor control in vision has tight feedback loops with vision itself. What I don’t believe is that we should model this as acting so as to minimize prediction loss. For one thing, I’ve read that a pretty good model of saccade movement patterns is that we look at the most surprising parts of the image, which would be better-modeled by moving eyes so as to maximize predictive loss.
Babies look longer at objects which they find surprising, as opposed to those which they recognize.
It’s true that PP can predict some behaviors like this, because you’d do this in order to learn, so that you minimize future prediction error. But that doesn’t mean PP is helping us predict those eye movements.
In a world dependent on money, a money-minimizing person might still have to obtain and use money in order to survive and get to a point where they can successfully do without money. That doesn’t mean we can look at money-seeking behavior and conclude that a person is a money-minimizer. More likely that they’re a money-maximizer. But they could be any number of things, because in this world, you have to deal with money in a broad variety of circumstances.
Let me briefly sketch an anti-PP theory. According to what you’ve said so far, I understand you as saying that we act in a way which minimizes prediction error, but according to a warped prior which doesn’t just try to model reality statistically accurately, but rather, increases the probability of things like food, sex, etc in accordance with their importance (to evolutionary fitness). This causes us to seek those things.
My anti-PP theory is this: we act in a way which maximizes prediction error, but according to a warped prior which doesn’t just model reality statistically accurately, but rather, decreases the probability of things like food, sex, etc in accordance with their importance. This causes us to seek those things.
I don’t particularly believe anti-PP, but I find it to be more plausible than PP. It fits human behavior better. It fits eye saccades better. (The eye hits surprising parts of the image, plus sexually significant parts of the image. It stands to reason that sexually significant images are artificially “surprising” to our visual system, making them more interesting.) It fits curiosity and play behavior better.
By the way, I’m actually much more amenable to the version of PP in Kaj Sotala’s post on craving, where warping epistemics by forcing belief in success is just one motivation among several in the brain. I do think something similar to that seems to happen, although my explanation for it is much different (see my earlier comment). I just don’t buy that this is the basic action mechanism of the brain, governing all our behavior, since it seems like a large swath of our behavior is basically the opposite of what you’d expect under this hypothesis. Yes, these predictions can always be fixed by sufficiently modifying the prior, forcing the “pursuing minimal prediction error” hypothesis to line up with the data we see. However, because humans are curious creatures who look at surprising things, engage in experimental play, and like to explore, you’re going to have to take a sensible probability distribution and just about reverse the probabilities to explain those observations. At that point, you might as well switch to anti-PP theory.
So, for your project of re-writing rationality in PP, would PP constitute a model of human irrationality, and how to rectify it, in contrast to ideal rationality (which would not be well-described by PP)?
Or would you employ PP both as a model which explains human irrationality and as an ideal rationality notion, so that we can use it both as the framework in which we describe irrationality and as the framework in which we can understand what better rationality would be?
Am I right in inferring from this that your preferred version of PP is one where we explicitly plan to minimize prediction error, as opposed to the Active Inference model (which instead minimizes KL divergence)? Or do you endorse an Active Inference type model?
This explanation in terms of evolution makes the PP theory consistent with observations, but does not give me a reason to believe PP. The added complexity to the prior is similar to the added complexity of other kinds of machinery to implement drives, so as yet I see no reason to prefer this explanation to other possibly explanations of what’s going on in the brain.
My remarks about problems with different versions of PP can each be patched in various ways; these are not supposed to be “gotcha” arguments in the sense of “PP can’t explain this! / PP can’t deal with this!”. Rather, I’m trying to boggle at why PP looks promising in the first place, as a hypothesis to raise to our attention.
Each of the arguments I mentioned are about one way I might see that someone might think PP is doing some work for us, and why I don’t see that as a promising avenue.
So I remain curious what the generators of your view are.