I broadly agree with the post on the Free Energy Principle, but I do think some clarifications are called for here, so I’ll do so:
For example, I’ll elaborate on what these quotes mean here:
It is widely accepted that FEP is an unfalsifiable tautology, including by proponents—see for example Beren Millidge, or Friston himself.
By the same token, once we find a computer-verified proof of any math theorem, we have revealed that it too is an unfalsifiable tautology. Even Fermat’s Last Theorem is now known to be a direct logical consequence of the axioms of math—arguably just a fancy way of writing 0=0.
So again, FEP is an unfalsifiable tautology. What does that mean in practice? Well, It means that I am entitled to never think about FEP. Anything that you can derive from FEP, you can derive directly from the same underlying premises from which FEP itself can be proven, without ever mentioning FEP.
(The underlying premises in question are something like “it’s a thing with homeostasis and bodily integrity”. So, by the way: if someone tells you “FEP implies organisms do X”, and if you can think of an extremely simple toy model of something with homeostasis & bodily integrity that doesn’t do X, then you can go tell that person that they’re wrong about what FEP implies!)
And this:
If somebody says “As a consequence of Hamiltonian mechanics, stars burn hydrogen”, we would correctly recognize that claim as nonsense. Hamiltonian mechanics applies to everything, whereas stars burning hydrogen is a specific contingent hypothesis that might or might not be true.
That’s how I feel when I read sentences like “In order to minimize free energy, the agent learns an internal model of potential states of the environment.” Maybe the agent does, or maybe it doesn’t! The free energy principle applies to all living things by definition, whereas “learning an internal model of potential states of the environment” is a specific hypothesis about an organism, a hypothesis that might be wrong. For example (as I mentioned in a different comment), imagine a very simple organism that evolved in an extremely, or even perfectly, homogeneous environment. This organism won’t evolve any machinery for learning (or storing or querying) an internal model of potential states of the environment, right? Sure, that’s not super realistic, but it’s certainly possible in principle. And if it happened, the FEP would apply to that organism just like every other organism, right? So the sentence above is just wrong. Or we can interpret the sentence more charitably as: “The free energy principle makes it obvious that it may be evolutionarily-adaptive for an organism to learn an internal model of potential states of the environment.” But then my response is: That was already obvious, without the FEP!
It’s definitely true that theories that apply to everything have little to nothing to say about their contents, and the reason for this is basically related to the conservation of expected evidence rule in Bayes’s theorem, which states that for every expectation of evidence, there is an equal and opposite expectation of counter-evidence, and if a theory doesn’t have that counter-evidence, it’s either a logical theory in which the tautologies are always true, or your theory is too general to explain something.
Thus, a theory that tries to apply to everything can’t really predict any empirical phenomenon (which is defined as a phenomenon that applies only sometimes, where there exists a possibility of not having it, and a possibility of having that.):
Jeff Beck does have an interesting counterargument, where the FEP/Active Inference is there to mathematically define agency and life from it’s complement/negation, and suggests a parallel to Russell’s paradox in the 19th century on why formalizing agency is necessary, because the mathematics/set theories people intuitively have often turn out to have inconsistencies, which is disastrous for any attempt at distinguishing true from false statements, and thus people work in formalized set theories like ZFC, which restrains the paradoxes of naive set theory, and this is caused by people not being logically ominisicent for computational reasons, and a Bayesian would never have logically inconsistent hypotheses in it’s sample space due to logical omnisicence.
More generally, I consider formalization for safety to be mostly not useful for AI safety, due to the timelines becoming shorter than they used to.
Finally, while I agree that “Changing your predictions to match the world” problem and “Changing the world to match your predictions” problem does have different algorithmic solutions in the brain, which makes it bad from a descriptive perspective, it’s still possible that the two domains are able to be unified more then they currently have, see here:
While I personally think FEP is way overrated by it’s practitioners, I do think some elements of their agenda are interesting enough to pursue as separate threads.
I broadly agree with the post on the Free Energy Principle, but I do think some clarifications are called for here, so I’ll do so:
For example, I’ll elaborate on what these quotes mean here:
And this:
It’s definitely true that theories that apply to everything have little to nothing to say about their contents, and the reason for this is basically related to the conservation of expected evidence rule in Bayes’s theorem, which states that for every expectation of evidence, there is an equal and opposite expectation of counter-evidence, and if a theory doesn’t have that counter-evidence, it’s either a logical theory in which the tautologies are always true, or your theory is too general to explain something.
Thus, a theory that tries to apply to everything can’t really predict any empirical phenomenon (which is defined as a phenomenon that applies only sometimes, where there exists a possibility of not having it, and a possibility of having that.):
Jeff Beck does have an interesting counterargument, where the FEP/Active Inference is there to mathematically define agency and life from it’s complement/negation, and suggests a parallel to Russell’s paradox in the 19th century on why formalizing agency is necessary, because the mathematics/set theories people intuitively have often turn out to have inconsistencies, which is disastrous for any attempt at distinguishing true from false statements, and thus people work in formalized set theories like ZFC, which restrains the paradoxes of naive set theory, and this is caused by people not being logically ominisicent for computational reasons, and a Bayesian would never have logically inconsistent hypotheses in it’s sample space due to logical omnisicence.
https://www.lesswrong.com/posts/MArdnet7pwgALaeKs/why-i-m-not-into-the-free-energy-principle#QaGuuYXPnpDqDQKCL
The issue is Markov blankets don’t actually work to formalize important stuff in the way that FEP people imagine, see here:
https://www.lesswrong.com/posts/vmfNaKbZ6urMdQrv2/agent-boundaries-aren-t-markov-blankets-unless-they-re-non
More generally, I consider formalization for safety to be mostly not useful for AI safety, due to the timelines becoming shorter than they used to.
Finally, while I agree that “Changing your predictions to match the world” problem and “Changing the world to match your predictions” problem does have different algorithmic solutions in the brain, which makes it bad from a descriptive perspective, it’s still possible that the two domains are able to be unified more then they currently have, see here:
https://www.lesswrong.com/posts/8dbimB7EJXuYxmteW/fixdt
While I personally think FEP is way overrated by it’s practitioners, I do think some elements of their agenda are interesting enough to pursue as separate threads.
I think it’s fine staying at 0.