Interesting. I found one paper that explains the one learning algorithm hypothesis and gives evidence for it. Quoting from it:
There seems to be some evidence for a single algorithm explaining the computations performed
by the primary auditory, visual, motor, and somatosensory cortices. Given how little is known
about higher-level processing immediately downstream of these primary areas, it is premature to
generalize to other areas located in the occipital lobe. Less is known about the details of the
computations performed in the areas located in anterior cortex and, in particular, the prefrontal
cortex, which is disproportionately enlarged in humans when compared to non-human primates.
Is there anything more up to date or comprehensive than this paper?
This tangent aside, I agree that it would be really valuable to improve the way we process evidence subconsciously. I’m a bit skeptical that you’ve actually found such a method, but I hope that you succeed in writing it down and that it really works.
Our Coalescing Minds paper had the one learning algorithm hypothesis as one of its assumptions; I wasn’t the neuroscience expert, but my co-author was, and here’s what he wrote about that premise (note that the paper was intended for a relatively popular audience, so the neuroscience detail was kept light):
An adult human neocortex consists of several areas which are to varying degrees specialized to process different types of information. The functional specialization is correlated with the anatomical differences of different cortical areas. Although there are obvious differences between areas, most cortical areas share many functional and anatomical traits. There has been considerable debate on whether cortical microcircuits are diverse or canonical [Buxhoeveden & Casanova, 2002; Nelson, 2002] but we argue that the differences are variations of the same underlying cortical algorithm, rather than entirely different algorithms. This is because most cortical areas seem to have the capability of processing any type of information. The differences seem to be a matter of optimization to a specific type of information, rather than a different underlying principle.
The cortical areas do lose much of their plasticity during maturation. For instance, it is possible to lose one’s ability to see colors if a specific visual cortical area responsible for color vision is damaged. The adult brain is not plastic enough to compensate for this damage, as the relevant regions have already specialized to their tasks. If the same brain regions were to be damaged during early childhood, color blindness would most likely not result.
However, this lack of plasticity reflects learning and specialization during the lifespan of the brain rather than innate algorithmic differences between different cortical areas. Plenty of evidence supports the idea that the different cortical areas can process any spatiotemporal patterns. For instance, the cortical area which normally receives auditory information and develops into the auditory cortex will develop visual representations if the axons carrying auditory information are surgically replaced by axons carrying visual information from the eyes [Newton & Sur, 2004]. The experiments were carried out with young kittens, but a somewhat similar sensory substitution is seen even in adult humans: relaying visual information through a tactile display mounted on the tongue will result in visual perception [Vuillerme & Cuisiner, 2009]. What first feels like tickling in the tongue will start feeling like seeing. In other words, the experience of seeing is not in the visual cortex but in the structure of the incoming information.
Another example of the mammalian brain’s ability to process any type of information is the development of trichromatic vision in mice that, like mammalian ancestors, normally have a dichromatic vision [Jacobs et al., 2007]. All it takes for a mouse to develop primate-like color vision is the addition of a gene encoding the photopigment which evolved in primates. When mice are born with this extra gene, their cortex is able to adapt to the new source information from the retina and to make sense of it. Even the adult cortical areas of humans can be surprisingly adaptive as long as the changes happen slowly enough [Feuillet et al., 2007]. Finally, Marzullo et al. [2010] demonstrated that rats implanted with electrodes both in their motor and visual cortices can learn to modulate the output from their motor cortex based on feedback given to visual cortex.
The paper you linked to about the one learning algorithm hypothesis is from 2012. Since that time the theory has gained significant strength from the advances in DL, and in particular the work on deep reinforcement learning. Proving that an ANN with a relatively simple initial/prior architecture and about 1 million neurons can reach human-level performance on a set of 100 games when trained end to end with RL is pretty strong (albeit indirect) evidence for the one learning hypothesis.
One key remaining question is then: how does the brain actually implement approximate optimization/learning that is at least as good as back-prop? We know that back-prop is not biologically realistic. On that front, Bengio’s group has made significant recent progress with a new technique/theory called target propagation 1, which originated in part as an explanation for how the brain could implement credit assignment, but it also shows promise as a potential replacement for backprop 2 - which further increases the biological plausibility.
In terms of more direct evidence, the hippocampus in particular appears to have a simple explanation in terms of reinforcement learning 3.
In terms of the prefrontal cortex in particular, there are working theories that explain much of the PFC as a set of modules specialized for working memory buffers that are controlled by gating units in the basal ganglia. That whole system in particular is also driven/learned through dopamine based RL.
Interesting. I found one paper that explains the one learning algorithm hypothesis and gives evidence for it. Quoting from it:
Is there anything more up to date or comprehensive than this paper?
This tangent aside, I agree that it would be really valuable to improve the way we process evidence subconsciously. I’m a bit skeptical that you’ve actually found such a method, but I hope that you succeed in writing it down and that it really works.
Our Coalescing Minds paper had the one learning algorithm hypothesis as one of its assumptions; I wasn’t the neuroscience expert, but my co-author was, and here’s what he wrote about that premise (note that the paper was intended for a relatively popular audience, so the neuroscience detail was kept light):
The paper you linked to about the one learning algorithm hypothesis is from 2012. Since that time the theory has gained significant strength from the advances in DL, and in particular the work on deep reinforcement learning. Proving that an ANN with a relatively simple initial/prior architecture and about 1 million neurons can reach human-level performance on a set of 100 games when trained end to end with RL is pretty strong (albeit indirect) evidence for the one learning hypothesis.
One key remaining question is then: how does the brain actually implement approximate optimization/learning that is at least as good as back-prop? We know that back-prop is not biologically realistic. On that front, Bengio’s group has made significant recent progress with a new technique/theory called target propagation 1, which originated in part as an explanation for how the brain could implement credit assignment, but it also shows promise as a potential replacement for backprop 2 - which further increases the biological plausibility.
In terms of more direct evidence, the hippocampus in particular appears to have a simple explanation in terms of reinforcement learning 3.
In terms of the prefrontal cortex in particular, there are working theories that explain much of the PFC as a set of modules specialized for working memory buffers that are controlled by gating units in the basal ganglia. That whole system in particular is also driven/learned through dopamine based RL.