For me, your examples of why visual perception needs the same things as language, including time window, is a standard, textbook-level (and often used!) proof of the fact they’re both (widely understood) Fodorian modules (in case of visual processing, two distinct modules indeed, though the labels “conscious” and “subconscious” are strange, I’m used to calling those “What-path” and “Where-path”), fine-tuned but not fully designed during the time-window, not that they are, vice versa, both handled by general algorithm like a snowflake.
Now, I understand that Fodorian modules (even when you throw away the old requirement of there being a strictly limited part of the cortex responsible for it) are not that widely held nowadays. However, when I look at people, I cannot help seeing them. From prosopagnosia to specific language impairments, aka aphasias (only two of the six commonly discussed aphasias are really language-based but the name stuck) to memory disruptions, we see individual modules breaking—including in-born reaking, before fine-tuning! - and just as well we see people whose general intelligence is reasonably low with unusually good performance of some of their modules.
Addendum: “visual” in “visual processing” is, of course, a red herring. It would be better to speak of two perception modules, with input variable (blindborn people fine-tune it to other things, for example—whereas those blinded in adulthood, AFAIK, do not).
your examples of why visual perception needs the same things as language, including time window, is a standard, textbook-level (and often used!) proof of the fact they’re both (widely understood) Fodorian modules
Interesting! Can you point me to a textbook or other reference that makes this argument?
the labels “conscious” and “subconscious” are strange, I’m used to calling those “What-path” and “Where-path”
If you’re talking about the dorsal and lateral streams, that’s not what I’m talking about. The dorsal steam and lateral stream are both part of the neocortex. I was talking about “vision processing in the neocortex” versus “vision processing in the midbrain (especially superior colliculus)”. Does that make sense?
On the second point—I have misunderstood you, now I see what you’re talking about. If Fodorian modules’ view is right, the neocortex one(s) still isn’t (aren’t) “conscious”. The received wisdom I have says that modules are:
1)Automatic (one cannot consciously change how they work—except by cutting off their input) - hence susceptible to illusions/wrong analyses/...;
2)Autonomous (consciousness only “sees” outputs, a module is black box for its owner; these two properties are related but distinct—yet something that has both can barely be called “conscious”);
3)Inherited with a critical period of fine-tuning (that’s basically what you called time window).
There were some more points but I (obviously) forgot them. And that brings me to your first point: I can’t point to a textbook right away but that was part of several courses I was taught (Psychology of cognitive processes at Moscow State University (Fundamental and Applied Linguistics program); Language, Music, and Cognition in NYI 2016 - nyi.spb.ru).
Thank you! I agree that much of what happens in the neocortex is not conscious, and agree with your 1+2+3. I have edited the wording of that sentence to be less confusing. As for what is and isn’t conscious, I like the Global Neuronal Workspace idea. I use that a lot when thinking about the neocortex.
I think I used the term “sensitive windows” in footnote 2 for what you call “critical period of fine-tuning”. I was thinking of things like low-level sensory processing, the fact that you can’t learn a language in the native accent as an adult, etc. Then I also talked about “time windows” in the context of filial imprinting.
I used two different terms, “sensitive windows” and “time windows”, but we may ask: are these two different things, or two examples of the same thing? I’m not sure. I would guess that they’re the same mechanism, but the former is locking in “normal” information-processing connections (from neocortex to neocortex), and the latter is locking in specifically connections to dopamine neurons or connections to other hormones or some other type of connection to the limbic system. I’m still trying to understand how those latter connections work...
I would think that the former are the _mechanism_ of the latter—though, as they say, “don’t quote me on that”.
There is an interesting question of whether, if many things are modules, there is also non-module part, the “general intelligence” part which does not share those properties. Perhaps unsurprisingly, there is no consensus (though my intuitions say there is the GI part).
Also, it seems that different modules might use the same (common) working memory—though this is not set in stone (and depends, in particular, on your analysis of language—if late Chomsky is right, only phonology (PF) and perhaps semantics (LF) are modular, whereas syntax uses our general recursive ability, and this is why it uses general working memory).
Specialized Functionality Via The Neocortex’s Gross Wiring Diagram—So if you seed a region with particular feedforward and feedback information streams, then it will find patterns and build up the corresponding concept space.
...and therefore, that there are lots of different neocortical modules exactly to the extent that there are lots of different neocortical regions, with strict borders between them and few axons crossing those borders. I’m not sure the extent to which this is the case.
Also, I’m not convinced that “general working memory” is a thing. My understanding is that we can do things like remember a word by putting it on loop using speech motor control circuits. But I would expect that someone with a lesion to those speech motor control circuits would still be able to, say, hold an image in their head for a short period of time using visual cortex. I think recurrent networks are ubiquitous in the neocortex, and any of those networks can potentially hold an activation at least for a couple seconds, and a few of them for much longer. Or maybe when you’re thinking about “general working memory”, you’re really thinking of the GNW?
1. “My understanding is that we can do things like remember a word by putting it on loop using speech motor control circuits”—this is called phonological loop in psycholinguistics (psychology) and is NOT THE SAME as working memory—in fact, tests for working memory usually include reading something aloud precisely to occupy the circuits and not let the test subject take advantage of their phonological loop. What I mean by working memory is the number of things one can hold in their mind simultaneously captured by “5+-2″ work and Daneman’s tests—whatever the explanation is.
2. Fodorian modules are, by definition, barely compatible with CCA. And the Zeitgeist of theoretical linguistics leads me to think that when you use RNN to explain something you’re cheating your way to performance instead of explaining what goes on (i.e. to think that brain ISN’T an RNN or a combination thereof—at least not in an obvious sense). Thus we don’t quite share neurological assumptions—though bridging to a common point may well be possible.
To be clear, I am using the term “recurrent” as a kinda generic term meaning “having a connectivity graph in which there are cycles”. That’s what I think is ubiquitous in the neocortex. I absolutely do not think that “the kind of RNN that ML practitioners frequently use today” is similar to how the neocortex works. Indeed, I think very few ML practitioners are using algorithms that are fundamentally similar to brain algorithms. (I think Dileep George is one of the exceptions.)
Fodorian modules are, by definition, barely compatible with CCA
...unless the Fodorian modules are each running the same algorithm on different input data, right?
Well, no. In particular, if you feed the same sound input to linguistic module (PF) and to the module of (say, initially visual) perception, the very intuition behind Fodorian modules is that they will *not* do the same—PF will try to find linguistic expressions similar to the input whereas the perception module will try to, well, tell where the sound comes from, how loud it is and things like that.
For me, your examples of why visual perception needs the same things as language, including time window, is a standard, textbook-level (and often used!) proof of the fact they’re both (widely understood) Fodorian modules (in case of visual processing, two distinct modules indeed, though the labels “conscious” and “subconscious” are strange, I’m used to calling those “What-path” and “Where-path”), fine-tuned but not fully designed during the time-window, not that they are, vice versa, both handled by general algorithm like a snowflake.
Now, I understand that Fodorian modules (even when you throw away the old requirement of there being a strictly limited part of the cortex responsible for it) are not that widely held nowadays. However, when I look at people, I cannot help seeing them. From prosopagnosia to specific language impairments, aka aphasias (only two of the six commonly discussed aphasias are really language-based but the name stuck) to memory disruptions, we see individual modules breaking—including in-born reaking, before fine-tuning! - and just as well we see people whose general intelligence is reasonably low with unusually good performance of some of their modules.
Addendum: “visual” in “visual processing” is, of course, a red herring. It would be better to speak of two perception modules, with input variable (blindborn people fine-tune it to other things, for example—whereas those blinded in adulthood, AFAIK, do not).
Interesting! Can you point me to a textbook or other reference that makes this argument?
If you’re talking about the dorsal and lateral streams, that’s not what I’m talking about. The dorsal steam and lateral stream are both part of the neocortex. I was talking about “vision processing in the neocortex” versus “vision processing in the midbrain (especially superior colliculus)”. Does that make sense?
On the second point—I have misunderstood you, now I see what you’re talking about. If Fodorian modules’ view is right, the neocortex one(s) still isn’t (aren’t) “conscious”. The received wisdom I have says that modules are:
1)Automatic (one cannot consciously change how they work—except by cutting off their input) - hence susceptible to illusions/wrong analyses/...;
2)Autonomous (consciousness only “sees” outputs, a module is black box for its owner; these two properties are related but distinct—yet something that has both can barely be called “conscious”);
3)Inherited with a critical period of fine-tuning (that’s basically what you called time window).
There were some more points but I (obviously) forgot them. And that brings me to your first point: I can’t point to a textbook right away but that was part of several courses I was taught (Psychology of cognitive processes at Moscow State University (Fundamental and Applied Linguistics program); Language, Music, and Cognition in NYI 2016 - nyi.spb.ru).
Thank you! I agree that much of what happens in the neocortex is not conscious, and agree with your 1+2+3. I have edited the wording of that sentence to be less confusing. As for what is and isn’t conscious, I like the Global Neuronal Workspace idea. I use that a lot when thinking about the neocortex.
I think I used the term “sensitive windows” in footnote 2 for what you call “critical period of fine-tuning”. I was thinking of things like low-level sensory processing, the fact that you can’t learn a language in the native accent as an adult, etc. Then I also talked about “time windows” in the context of filial imprinting.
I used two different terms, “sensitive windows” and “time windows”, but we may ask: are these two different things, or two examples of the same thing? I’m not sure. I would guess that they’re the same mechanism, but the former is locking in “normal” information-processing connections (from neocortex to neocortex), and the latter is locking in specifically connections to dopamine neurons or connections to other hormones or some other type of connection to the limbic system. I’m still trying to understand how those latter connections work...
I would think that the former are the _mechanism_ of the latter—though, as they say, “don’t quote me on that”.
There is an interesting question of whether, if many things are modules, there is also non-module part, the “general intelligence” part which does not share those properties. Perhaps unsurprisingly, there is no consensus (though my intuitions say there is the GI part).
Also, it seems that different modules might use the same (common) working memory—though this is not set in stone (and depends, in particular, on your analysis of language—if late Chomsky is right, only phonology (PF) and perhaps semantics (LF) are modular, whereas syntax uses our general recursive ability, and this is why it uses general working memory).
Hmm, interesting!
My thinking about general intelligence is the combination of
Common Cortical Algorithm—every part of the neocortex is a similarly-constructed general-purpose generative model builder that basically does some combination of self-supervised learning, RL, and Model Predictive Control
Specialized Functionality Via The Neocortex’s Gross Wiring Diagram—So if you seed a region with particular feedforward and feedback information streams, then it will find patterns and build up the corresponding concept space.
...and therefore, that there are lots of different neocortical modules exactly to the extent that there are lots of different neocortical regions, with strict borders between them and few axons crossing those borders. I’m not sure the extent to which this is the case.
Global Neuronal Workspace (GNW) connecting diverse parts of the neocortex to each other and to long-term memory.
“System 2” is just a way of stringing together various automatic thoughts (generative models) into a sorta ad-hoc serial computer.
Also, I’m not convinced that “general working memory” is a thing. My understanding is that we can do things like remember a word by putting it on loop using speech motor control circuits. But I would expect that someone with a lesion to those speech motor control circuits would still be able to, say, hold an image in their head for a short period of time using visual cortex. I think recurrent networks are ubiquitous in the neocortex, and any of those networks can potentially hold an activation at least for a couple seconds, and a few of them for much longer. Or maybe when you’re thinking about “general working memory”, you’re really thinking of the GNW?
1. “My understanding is that we can do things like remember a word by putting it on loop using speech motor control circuits”—this is called phonological loop in psycholinguistics (psychology) and is NOT THE SAME as working memory—in fact, tests for working memory usually include reading something aloud precisely to occupy the circuits and not let the test subject take advantage of their phonological loop. What I mean by working memory is the number of things one can hold in their mind simultaneously captured by “5+-2″ work and Daneman’s tests—whatever the explanation is.
2. Fodorian modules are, by definition, barely compatible with CCA. And the Zeitgeist of theoretical linguistics leads me to think that when you use RNN to explain something you’re cheating your way to performance instead of explaining what goes on (i.e. to think that brain ISN’T an RNN or a combination thereof—at least not in an obvious sense). Thus we don’t quite share neurological assumptions—though bridging to a common point may well be possible.
Thank you for clearing up my confusion! :-)
To be clear, I am using the term “recurrent” as a kinda generic term meaning “having a connectivity graph in which there are cycles”. That’s what I think is ubiquitous in the neocortex. I absolutely do not think that “the kind of RNN that ML practitioners frequently use today” is similar to how the neocortex works. Indeed, I think very few ML practitioners are using algorithms that are fundamentally similar to brain algorithms. (I think Dileep George is one of the exceptions.)
...unless the Fodorian modules are each running the same algorithm on different input data, right?
Oh, then sorry about the RNN attack ;)
Well, no. In particular, if you feed the same sound input to linguistic module (PF) and to the module of (say, initially visual) perception, the very intuition behind Fodorian modules is that they will *not* do the same—PF will try to find linguistic expressions similar to the input whereas the perception module will try to, well, tell where the sound comes from, how loud it is and things like that.