But more seriously, this is basically how evolution works too. It starts with a simple system and then it patches it. Evolved systems are messy and convoluted.
I don’t deny this. My fear isn’t a general fear that any time we conclude there’s a base system with some patches, we’re wrong. Rather, I have a fear of using these patches to excuse a bad theory, like epicycle theory vs Newton. The specific worry is more like why do people start buying this in the first place? I’ve never seen concrete evidence that it helps people understand things?? And when people check the math in Friston papers, it seems to be a Swiss Cheese of errors???
If you think I’m wrong, then you can register your counter-prediction and we can check back in 30 years and we’ll see if one of us has been proven right.
To state the obvious, this feedback loop is too slow, but obviously that’s compatible with your point here.
Still, I hope we can find predictions that can be tested faster.
Or even moreso, I hope that we can spell out reasons for believing things which help us find double-cruxes which we can settle through simple discussion.
Treating “PP” as a monolithic ideology probably greatly exaggerates the seeming disagreement. I don’t have any dispute with a lot of the concrete PP methodology. For example, the predictive coding = gradient descent paper commits no sins by my lights. I haven’t understood the math in enough detail to believe the biological implications yet (I feel, uneasily, like there might be a catch somewhere which makes it still not too biologically plausible). But at base, it’s a result showing that a specific variational method is in-some-sense equivalent to gradient descent.
(As long as we’re in the realm of “some specific variational method” instead of blurring everything together into “free energy minimization”, I’m relatively happier.)
If you want to get into that level of technical granularity then there are major things that need to change before applying the PP methodology in the paper to real biological neurons. Two of the big ones are brainwave oscillations and existing in the flow of time.
Mostly what I find interesting is the theory that the bulk of animal brain processing goes into creating a real-time internal simulation of the world, that this is mathematically plausible via forward-propagating signals, and that error and entropy are fused together.
When I say “free energy minimization” I mean the idea that error and surprise are fused together (possibly with an entropy minimizer thrown in).
I don’t deny this. My fear isn’t a general fear that any time we conclude there’s a base system with some patches, we’re wrong. Rather, I have a fear of using these patches to excuse a bad theory, like epicycle theory vs Newton. The specific worry is more like why do people start buying this in the first place? I’ve never seen concrete evidence that it helps people understand things?? And when people check the math in Friston papers, it seems to be a Swiss Cheese of errors???
To state the obvious, this feedback loop is too slow, but obviously that’s compatible with your point here.
Still, I hope we can find predictions that can be tested faster.
Or even moreso, I hope that we can spell out reasons for believing things which help us find double-cruxes which we can settle through simple discussion.
Treating “PP” as a monolithic ideology probably greatly exaggerates the seeming disagreement. I don’t have any dispute with a lot of the concrete PP methodology. For example, the predictive coding = gradient descent paper commits no sins by my lights. I haven’t understood the math in enough detail to believe the biological implications yet (I feel, uneasily, like there might be a catch somewhere which makes it still not too biologically plausible). But at base, it’s a result showing that a specific variational method is in-some-sense equivalent to gradient descent.
(As long as we’re in the realm of “some specific variational method” instead of blurring everything together into “free energy minimization”, I’m relatively happier.)
If you want to get into that level of technical granularity then there are major things that need to change before applying the PP methodology in the paper to real biological neurons. Two of the big ones are brainwave oscillations and existing in the flow of time.
Mostly what I find interesting is the theory that the bulk of animal brain processing goes into creating a real-time internal simulation of the world, that this is mathematically plausible via forward-propagating signals, and that error and entropy are fused together.
When I say “free energy minimization” I mean the idea that error and surprise are fused together (possibly with an entropy minimizer thrown in).