Maybe you’re ahead of me, but it took me until long after this post—just a couple weeks ago—to realize that you can take a neural circuit set up for RL, and jury-rig it to do supervised learning instead.
I think this is a big part of the story behind what the vmPFC is doing. And, in a certain sense, the amygdala too. More on this in a forthcoming post.
couldn’t the neocortex also be running lookup table like auto or hetero-associative learning … Why is the cerebellum faster at producing outputs than the neocortex?
I think of the neocortex as doing analysis-by-synthesis—it searches through a space of generative models for one that matches the input data. There’s a lot of recurrency—the signals bounce around until it settles into an equilibrium. For example in this model, there’s a forward pass from the input data, and that activates some generative models that seem plausible. But it may be multiple models that are mutually inconsistent. For example, in the vision system, “Yarn” and “Yam” are sufficiently close that a feedforward pass would activate both possibilities simultaneously. Then there’s this message-passing algorithm where the different possibilities compete to explain the data, and it settles on one particular compositional generative model.
So this seems like a generally pretty slow inference algorithm. But the slowness is worth it, because the neocortex winds up understanding the input data, i.e. fitting it into its structured model of the world, and hence it can now flexibly query it, make predictions, etc.
I think the cerebellum is much simpler than that, and closer to an actual lookup table, and hence presumably much faster.
The cerebellum is also closer in proximity to the spinal cord, which reduces communication delays when reading proprioceptive nerves and commanding muscles.
That paper says “it has been the dominant working assumption in the field that the CS is transmitted to the cerebellar cortex via the mossy fibres (mf) and parallel fibres (pf) whereas information about the US is provided by climbing fibres (cf) originating in the inferior olive”, which is what I said (US=shoulda, CS=context). Where are you disagreeing here? Or do they contradict that later in the paper?
How does the neocortex carry the “shoulda” signal?
The neocortex is a fancy algorithm that understands the world and takes intelligent actions. It’s not perfect, but it’s pretty great! So whatever the neocortex does, that’s kinda a “ground truth” for the question of “What is the right thing to do right now?”—at least, it’s a ground truth from the perspective of the much stupider cerebellum. So my proposal is that whatever the neocortex is outputting, that’s a “shoulda” signal that the cerebellum wants to imitate.
This suggests that the neocortex can learn the cerebellar mapping and short-circuit to use it? Why does it need to go through the cerebellum to do this? Rather than via the motor cortex and efferent connections back to the muscles?
I’m not sure I understand your question. The neocortex outputs don’t need to go through the cerebellum. People can be born without a cerebellum entirely, they turn out OK. But since the cerebellum is like a super-fast memoizer / lookup table, I think the neocortex can work better by passing signals through the cerebellum.
Anyway, that was all just casual speculation, I don’t know how the motor cortex, midbrain, cerebellum, and outgoing nerves are wired together. (I’m very interested to learn, just haven’t gotten around to it.)
Hmm, this page suggests that there are both motor-pathways-through-the-cerebellum and motor-pathways-bypassing-the-cerebellum. But that’s not a reliable source—that page seems to be riddled with errors. So I dunno.
Thanks for the great questions!!
Maybe you’re ahead of me, but it took me until long after this post—just a couple weeks ago—to realize that you can take a neural circuit set up for RL, and jury-rig it to do supervised learning instead.
I think this is a big part of the story behind what the vmPFC is doing. And, in a certain sense, the amygdala too. More on this in a forthcoming post.
I think of the neocortex as doing analysis-by-synthesis—it searches through a space of generative models for one that matches the input data. There’s a lot of recurrency—the signals bounce around until it settles into an equilibrium. For example in this model, there’s a forward pass from the input data, and that activates some generative models that seem plausible. But it may be multiple models that are mutually inconsistent. For example, in the vision system, “Yarn” and “Yam” are sufficiently close that a feedforward pass would activate both possibilities simultaneously. Then there’s this message-passing algorithm where the different possibilities compete to explain the data, and it settles on one particular compositional generative model.
So this seems like a generally pretty slow inference algorithm. But the slowness is worth it, because the neocortex winds up understanding the input data, i.e. fitting it into its structured model of the world, and hence it can now flexibly query it, make predictions, etc.
I think the cerebellum is much simpler than that, and closer to an actual lookup table, and hence presumably much faster.
The cerebellum is also closer in proximity to the spinal cord, which reduces communication delays when reading proprioceptive nerves and commanding muscles.
That paper says “it has been the dominant working assumption in the field that the CS is transmitted to the cerebellar cortex via the mossy fibres (mf) and parallel fibres (pf) whereas information about the US is provided by climbing fibres (cf) originating in the inferior olive”, which is what I said (US=shoulda, CS=context). Where are you disagreeing here? Or do they contradict that later in the paper?
The neocortex is a fancy algorithm that understands the world and takes intelligent actions. It’s not perfect, but it’s pretty great! So whatever the neocortex does, that’s kinda a “ground truth” for the question of “What is the right thing to do right now?”—at least, it’s a ground truth from the perspective of the much stupider cerebellum. So my proposal is that whatever the neocortex is outputting, that’s a “shoulda” signal that the cerebellum wants to imitate.
I’m not sure I understand your question. The neocortex outputs don’t need to go through the cerebellum. People can be born without a cerebellum entirely, they turn out OK. But since the cerebellum is like a super-fast memoizer / lookup table, I think the neocortex can work better by passing signals through the cerebellum.
Anyway, that was all just casual speculation, I don’t know how the motor cortex, midbrain, cerebellum, and outgoing nerves are wired together. (I’m very interested to learn, just haven’t gotten around to it.)
Hmm, this page suggests that there are both motor-pathways-through-the-cerebellum and motor-pathways-bypassing-the-cerebellum. But that’s not a reliable source—that page seems to be riddled with errors. So I dunno.