I think Friston’s free energy principle has a lot to offer here in terms of generalizing agency to include everything from amoebas to humans (although, ironically, maybe not AIXI).
Basically, rational agents, and living systems more generally, are built to regulate themselves in resistance to entropy by minimizing the free energy (essentially, “conflict”) between their a priori models of themselves as living or rational systems and what they actually experience through their senses. To do this well, they need to have some sort of internal model of their environment (https://en.m.wikipedia.org/wiki/Good_regulator) that they use for responding to changes in what they sense in order to increase the likelihood of their survival.
For human minds, this internal model is encoded in the neural circuitry of our brains. For amoebas, this internal model could be encoded in the states of the promoters and inhibitors in its gene regulatory network.
I am having trouble understanding the “free energy principle” being anything more than a control system that tries to minimize prediction error. If that’s all that is, there is nothing special about living systems, engineers have been building control systems for a long time. By that definition a Boston Dynamics walking robot is definitely a living system...
That’s not unreasonable as a quick summary of the principle.
I would say there is more to what makes a living system alive than just following the free energy principle per se. For instance, the robot would also need to scavenge for material and energy resources to incorporate into itself for maintenance, repair, and/or reproduction. Just correcting its gait when thrown off balance allows it minimize a sort of behavioral free energy, but that’s not enough to count as alive.
But if you want to put amoebas and humans in the same qualitative category of “agency”, then you need a framework that is general enough to capture the commonalities of interest. And yes, under such a broad umbrella, artificial control systems and dynamically balancing walking robots would be included.
The free energy principle applies to a lot of systems, not just living or agentic. I see it more as a way to systematize our approach to understanding a system or process rather than an explanation in and of itself. By focusing on how a system maintains set points (e.g., homeostasis) and minimizes prediction error (e.g., unsupervised learning), I think we would be better positioned to figure out what real agents are actually doing in a way that could inform the both the design and alignment of AGI.
To be honest, when I talk about the “free energy principle”, I typically have in mind a certain class of algorithmic implementations of it, involving generative models and using maximum likelihood estimation through online gradient descent to minimize their prediction errors. Something like https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000211
I’ve never been able to make sense of the “Good Regulator” theorem, and in the original paper (linked in that Wikipedia article) I cannot map their terminology onto any control system I can think of. Can you explain it? Because it seems obvious to me that a room thermostat contains no model of anything, and I can find no way of mapping the components of that system to Conant and Ashby’s terminology. Their own example of a hunter shooting pheasants is just as opaque to me.
The model may be implicit, but it’s embedded in the structure of the whole thermostat system, from the thermometer that measures temperature to the heating and cooling systems that it controls. For instance, it “knows” that turning on the heat is the appropriate thing to do when the temperature it reads falls below its set point. There is an implication there that the heater causes temperature to rise, or that the AC causes it to fall, even though it’s obviously not running simulations (unless it’s a really good thermostat) on how the heating/cooling systems affect the dynamics of temperature fluctuations in the building.
The engineers did all the modeling beforehand, then built the thermostat to activate the heating and cooling systems in response to temperature fluctuations according to the rules that they precomputed. Evolution did just this in building the structure of the amoeba’s gene networks and the suite of human instincts (heritable variation + natural selection is how information is transferred from the environment into a species’ genome). Lived experience pushes further information from the environment to the internal model, upregulating or downregulating various genes in response to stimuli or learning to reinforce certain behaviors in certain contexts. But environmental information was already there in the structure to begin with, just like it is in more traditional artificial control systems.
The example with the hunter and pheasants was just to show how “regulating” (i.e., consistently achieving a state in the desirable set = pheasant successfully shot) requires the hunter to have a good mental model of the system (pheasant behavior, wind disturbances, etc.). Again, this model does not have to be explicit in general but could be completely innate.
I can’t match any of that up to Conant and Ashby’s paper, though.
As you say, the engineers designing a thermostat have a model of the system. But the thermostat does not. It simply compares the temperature with that set on the dial and turns a switch on and off. There is no trace of any model, prediction, expectation, knowledge of what its actions do, and so on. The engineers do have these things, the proof of which is that you can elicit their knowledge. The thermostat does not, the proof of which is that nowhere in the thermostat can any of these things be found.
The hunter is an obscure example, because no-one knows how humans accomplish such things, and instead we mostly make up stories based on what the process feels like from within. This method has a poor track record. More illuminating would be to look at a similar but man-made system: an automatic anti-aircraft gun shooting at a plane. Whether the control systems inside this device contain models is an empirical question, to be answered by looking at how it works. Maybe it does, and maybe it doesn’t. There is such a thing as model-based control, and there is such a thing as PID controllers (which do not contain models).
I think Friston’s free energy principle has a lot to offer here in terms of generalizing agency to include everything from amoebas to humans (although, ironically, maybe not AIXI).
Basically, rational agents, and living systems more generally, are built to regulate themselves in resistance to entropy by minimizing the free energy (essentially, “conflict”) between their a priori models of themselves as living or rational systems and what they actually experience through their senses. To do this well, they need to have some sort of internal model of their environment (https://en.m.wikipedia.org/wiki/Good_regulator) that they use for responding to changes in what they sense in order to increase the likelihood of their survival.
For human minds, this internal model is encoded in the neural circuitry of our brains. For amoebas, this internal model could be encoded in the states of the promoters and inhibitors in its gene regulatory network.
I am having trouble understanding the “free energy principle” being anything more than a control system that tries to minimize prediction error. If that’s all that is, there is nothing special about living systems, engineers have been building control systems for a long time. By that definition a Boston Dynamics walking robot is definitely a living system...
That’s not unreasonable as a quick summary of the principle.
I would say there is more to what makes a living system alive than just following the free energy principle per se. For instance, the robot would also need to scavenge for material and energy resources to incorporate into itself for maintenance, repair, and/or reproduction. Just correcting its gait when thrown off balance allows it minimize a sort of behavioral free energy, but that’s not enough to count as alive.
But if you want to put amoebas and humans in the same qualitative category of “agency”, then you need a framework that is general enough to capture the commonalities of interest. And yes, under such a broad umbrella, artificial control systems and dynamically balancing walking robots would be included.
The free energy principle applies to a lot of systems, not just living or agentic. I see it more as a way to systematize our approach to understanding a system or process rather than an explanation in and of itself. By focusing on how a system maintains set points (e.g., homeostasis) and minimizes prediction error (e.g., unsupervised learning), I think we would be better positioned to figure out what real agents are actually doing in a way that could inform the both the design and alignment of AGI.
To be honest, when I talk about the “free energy principle”, I typically have in mind a certain class of algorithmic implementations of it, involving generative models and using maximum likelihood estimation through online gradient descent to minimize their prediction errors. Something like https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000211
I’ve never been able to make sense of the “Good Regulator” theorem, and in the original paper (linked in that Wikipedia article) I cannot map their terminology onto any control system I can think of. Can you explain it? Because it seems obvious to me that a room thermostat contains no model of anything, and I can find no way of mapping the components of that system to Conant and Ashby’s terminology. Their own example of a hunter shooting pheasants is just as opaque to me.
The model may be implicit, but it’s embedded in the structure of the whole thermostat system, from the thermometer that measures temperature to the heating and cooling systems that it controls. For instance, it “knows” that turning on the heat is the appropriate thing to do when the temperature it reads falls below its set point. There is an implication there that the heater causes temperature to rise, or that the AC causes it to fall, even though it’s obviously not running simulations (unless it’s a really good thermostat) on how the heating/cooling systems affect the dynamics of temperature fluctuations in the building.
The engineers did all the modeling beforehand, then built the thermostat to activate the heating and cooling systems in response to temperature fluctuations according to the rules that they precomputed. Evolution did just this in building the structure of the amoeba’s gene networks and the suite of human instincts (heritable variation + natural selection is how information is transferred from the environment into a species’ genome). Lived experience pushes further information from the environment to the internal model, upregulating or downregulating various genes in response to stimuli or learning to reinforce certain behaviors in certain contexts. But environmental information was already there in the structure to begin with, just like it is in more traditional artificial control systems.
The example with the hunter and pheasants was just to show how “regulating” (i.e., consistently achieving a state in the desirable set = pheasant successfully shot) requires the hunter to have a good mental model of the system (pheasant behavior, wind disturbances, etc.). Again, this model does not have to be explicit in general but could be completely innate.
I can’t match any of that up to Conant and Ashby’s paper, though.
As you say, the engineers designing a thermostat have a model of the system. But the thermostat does not. It simply compares the temperature with that set on the dial and turns a switch on and off. There is no trace of any model, prediction, expectation, knowledge of what its actions do, and so on. The engineers do have these things, the proof of which is that you can elicit their knowledge. The thermostat does not, the proof of which is that nowhere in the thermostat can any of these things be found.
The hunter is an obscure example, because no-one knows how humans accomplish such things, and instead we mostly make up stories based on what the process feels like from within. This method has a poor track record. More illuminating would be to look at a similar but man-made system: an automatic anti-aircraft gun shooting at a plane. Whether the control systems inside this device contain models is an empirical question, to be answered by looking at how it works. Maybe it does, and maybe it doesn’t. There is such a thing as model-based control, and there is such a thing as PID controllers (which do not contain models).