If the cortical algorithm is the same across all mammals, why do only humans develop complex language? Do you think that the human neocortex is specialized for language in some way, or do you think that other mammal’s neocortices would be up to the task if sufficiently scaled up? What about our subcortex—do we get special language-based rewards? How would the subcortex implement those?
Furthermore, there are lots of commonalities across human languages—e.g. word order patterns and grammar similarities, see e.g. linguistic universals—how does that make sense if language is neocortical and the neocortex is a blank slate? Do linguistic commonalities come from the subcortex, from our shared environment, or from some way in which our neocortex is predisposed to learn language?
Also, on a completely different note, in asking “how does the subcortex steer the neocortex?” you seem to presuppose that the subcortex actually succeeds in steering the neocortex—how confident in that should we be? It seems like there are lots of things that people do that go against a naive interpretation of the subcortical reward algorithm—abstaining from sex, for example, or pursuing complex moral theories like utilitarianism. If the way that the subcortex steers the neocortex is terrible and just breaks down off-distribution, then that sort of cuts into your argument that we should be focusing on understanding how the subcortex steers the neocortex, since if it’s not doing a very good job then there’s little reason for us to try and copy it.
Here’s what I’m thinking: (1) I expect that the subcortex has an innate “human speech sound” detector, and tells the neocortex that this is an important thing to model; (2) maybe some adjustment of the neocortex information flows and hyperparameters, although I couldn’t tell you how. (I haven’t dived into the literature in either case.)
I do now have some intuition that some complicated domains may require some micromanagement of the learning process … in particular in this paper they found that to get vision to develop in their models, it was important that first they set up connections between low-level visual information and blah blah, and after learning those relationships, then they also connect the low-level visual information to some other information stream, and it can learn those relationships. If they just connect all the information streams at once, then the algorithm would flail around and not learn anything useful. It’s possible that vision is unusually complicated. Or maybe it’s similar for language: maybe there’s a convoluted procedure necessary to reliably get the right low-level model space set up for language. For example, I hear that some kids are very late talkers, but when they start talking, it’s almost immediately in full sentences. Is that a sign of some new region-to-region connection coming online in a carefully-choreographed developmental sequence? Maybe it’s in the literature somewhere, I haven’t looked. Just thinking out loud.
linguistic universals
I would say: the neocortical algorithm is built on certain types of data structures, and certain ways of manipulating and combining those data structures. Languages have to work smoothly with those types of data structures and algorithmic processes. In fact, insofar as there are linguistic universals (the wiki article says it’s controversial; I wouldn’t know either way), perhaps studying them might shed light on how the neocortical algorithm works!
you seem to presuppose that the subcortex actually succeeds in steering the neocortex
That’s a fair point.
My weak answer is: however it does its thing, we might as well try to understand it. They can be tools in our toolbox, and a starting point for further refinement and engineering. (ETA: …And if we want to make an argument that We’re Doomed if people reverse-engineer neocortical algorithms, which I consider a live possibility, it seems like understanding the subcortex would be a necessary part of making that argument.)
My more bold answer is: Hey, maybe this really would solve the problem! This seems to be a path to making an AGI which cares about people to the same extent and for exactly the same underlying reasons as people care about other people. After all, we would have the important ingredients in the algorithm, we can feed it the right memes, etc. In fact, we can presumably do better than “intelligence-amplified normal person” by twiddling the parameters in the algorithm—less jealousy, more caution, etc. I guess I’m thinking of Eliezer’s statement here that he’s “pretty much okay with somebody giving [Paul Christiano or Carl Shulman] the keys to the universe”. So maybe the threshold for success is “Can we make an AGI which is at least as wise and pro-social as Paul Christiano or Carl Shulman?”… In which case, there’s an argument that we are likely to succeed if we can reverse-engineer key parts of the neocortex and subcortex.
(I’m putting that out there, but I haven’t thought about it very much. I can think of possible problems. What if you need a human body for the algorithms to properly instill prosociality? What if there’s a political campaign to make the systems “more human” including putting jealousy and self-interest back in? If we cranked up the intelligence of a wise and benevolent human, would they remain wise and benevolent forever? I dunno...)
Some things which don’t fully make sense to me:
If the cortical algorithm is the same across all mammals, why do only humans develop complex language? Do you think that the human neocortex is specialized for language in some way, or do you think that other mammal’s neocortices would be up to the task if sufficiently scaled up? What about our subcortex—do we get special language-based rewards? How would the subcortex implement those?
Furthermore, there are lots of commonalities across human languages—e.g. word order patterns and grammar similarities, see e.g. linguistic universals—how does that make sense if language is neocortical and the neocortex is a blank slate? Do linguistic commonalities come from the subcortex, from our shared environment, or from some way in which our neocortex is predisposed to learn language?
Also, on a completely different note, in asking “how does the subcortex steer the neocortex?” you seem to presuppose that the subcortex actually succeeds in steering the neocortex—how confident in that should we be? It seems like there are lots of things that people do that go against a naive interpretation of the subcortical reward algorithm—abstaining from sex, for example, or pursuing complex moral theories like utilitarianism. If the way that the subcortex steers the neocortex is terrible and just breaks down off-distribution, then that sort of cuts into your argument that we should be focusing on understanding how the subcortex steers the neocortex, since if it’s not doing a very good job then there’s little reason for us to try and copy it.
Thanks!!
Here’s what I’m thinking: (1) I expect that the subcortex has an innate “human speech sound” detector, and tells the neocortex that this is an important thing to model; (2) maybe some adjustment of the neocortex information flows and hyperparameters, although I couldn’t tell you how. (I haven’t dived into the literature in either case.)
I do now have some intuition that some complicated domains may require some micromanagement of the learning process … in particular in this paper they found that to get vision to develop in their models, it was important that first they set up connections between low-level visual information and blah blah, and after learning those relationships, then they also connect the low-level visual information to some other information stream, and it can learn those relationships. If they just connect all the information streams at once, then the algorithm would flail around and not learn anything useful. It’s possible that vision is unusually complicated. Or maybe it’s similar for language: maybe there’s a convoluted procedure necessary to reliably get the right low-level model space set up for language. For example, I hear that some kids are very late talkers, but when they start talking, it’s almost immediately in full sentences. Is that a sign of some new region-to-region connection coming online in a carefully-choreographed developmental sequence? Maybe it’s in the literature somewhere, I haven’t looked. Just thinking out loud.
I would say: the neocortical algorithm is built on certain types of data structures, and certain ways of manipulating and combining those data structures. Languages have to work smoothly with those types of data structures and algorithmic processes. In fact, insofar as there are linguistic universals (the wiki article says it’s controversial; I wouldn’t know either way), perhaps studying them might shed light on how the neocortical algorithm works!
That’s a fair point.
My weak answer is: however it does its thing, we might as well try to understand it. They can be tools in our toolbox, and a starting point for further refinement and engineering. (ETA: …And if we want to make an argument that We’re Doomed if people reverse-engineer neocortical algorithms, which I consider a live possibility, it seems like understanding the subcortex would be a necessary part of making that argument.)
My more bold answer is: Hey, maybe this really would solve the problem! This seems to be a path to making an AGI which cares about people to the same extent and for exactly the same underlying reasons as people care about other people. After all, we would have the important ingredients in the algorithm, we can feed it the right memes, etc. In fact, we can presumably do better than “intelligence-amplified normal person” by twiddling the parameters in the algorithm—less jealousy, more caution, etc. I guess I’m thinking of Eliezer’s statement here that he’s “pretty much okay with somebody giving [Paul Christiano or Carl Shulman] the keys to the universe”. So maybe the threshold for success is “Can we make an AGI which is at least as wise and pro-social as Paul Christiano or Carl Shulman?”… In which case, there’s an argument that we are likely to succeed if we can reverse-engineer key parts of the neocortex and subcortex.
(I’m putting that out there, but I haven’t thought about it very much. I can think of possible problems. What if you need a human body for the algorithms to properly instill prosociality? What if there’s a political campaign to make the systems “more human” including putting jealousy and self-interest back in? If we cranked up the intelligence of a wise and benevolent human, would they remain wise and benevolent forever? I dunno...)