I was focusing on the implementation of a particular aspect of that. Specifically, when you’re doing what you call “thing modeling”, the “things” you wind up with are entries in a complicated learned world-model—e.g. “thing #6564457” is a certain horrifically complicated statistical regularity in multimodal sensory data, something like: “thing #6564457“ is a prediction that thing #289347 is present, and thing #89672, and thing #68972, but probably not thing #903672”, or whatever.
Meanwhile I agree with you that there is some brainstem / hypothalamus function (outside the learned world-model) that can evaluate how biologically adaptive it would be to eat food with a certain profile of flavors / smells / etc., given the current readings of various sensors detecting nutrient deficiencies etc. (That component actually seems quite straightforward to me.)
And then my particular focus is how exactly the brain builds interface into and out of the world-model, which is a prerequisite for learning that this particular statistical regularity (in the learned world-model) corresponds to a particular vector of sweetness, savoriness, etc. (in the brainstem), which the brainstem can analyze and declare likely to satisfy current physiological needs, and then therefore let’s try to eat it (back in the learned world-model).
If you look closely at what you wrote, I think you’ll find a few places where you need to transfer information into and out of the learned world-model. That’s fine, but there has to be a way that that works, and that’s the part I was especially interested in.
I guess my underlying assumption is that this interfacing isn’t a trivial thing—like I don’t think you can just casually say “The world-model shall have an item type called ‘food’ in it” and then there’s an obvious way to make that happen. I think the world-model is built from the ground up, as learned patterns, and patterns in the patterns, etc., so you generally need stories for how things are learned. At any rate, that type of algorithm seems plausible to me (from everything I’ve read and thought about), and I haven’t seen any alternative story that makes sense to me so far.
When the vector of various kinds of “hunger levels” changes, a relatively hard coded circuit probably exists that (abstractly (maybe even literally?)) assigns each food a new dot product value in terms of general food goodness, and foods with high values “sort to the top”, after which a whole planning engine kicks in, confabulating plans for getting any or all such high value foods and throwing out the implausible plans, until a food with a high value and a plausible plan is left over.
It sounds like you want to start with the hypothalamus & brainstem providing a ranked list of all possible foods, and then the world-model finds one that can be eaten. But I want to go basically in the opposite direction, where the planner (working within the learned world-model) proposes a thought that involves eating (“I could eat that carrot in the fridge”), and the hypothalamus & brainstem evaluate how appealing that plan is (“carrots are sweet, I’m low in sugar right now, 7/10″), and then it sends dopamine to reward the thinking of appealing thoughts (and moreso if they’re part of realistic likely-to-succeed plans). Like, if I’m really hungry, I think I’m more likely to decide to eat the first easily-accessible food that pops into my head, rather than working my way through a long list of possible foods that would be hypothetically better (“no I don’t have a coconut smoothie, no I don’t have fried clams, …”). Then, through the magic of reinforcement learning, the planner gradually learns to skillfully and quickly come up with appropriate and viable foods.
Response appreciated! Yeah. I think I have two hunches here that cause me to speak differently.
One of these hunches is that hunger sensors are likely to be very very “low level”, and motivationally “primary”. The other is maybe an expectation that almost literally “every possible thought” is being considered simultaneously in the brain by default, but most do not rise to awareness, or action-production, or verbalizability?
Like I think that hunger sensors firing will cause increased firing in “something or other” that sort of “represents food” (plus giving the food the halo of temporary desirability) and I expect this firing rate to basically go up over time… more hungriness… more food awareness?
Like if you ask me “Jennifer, when was the last time you ate steak?” then I am aware of a wave of candidate answers, and many fall away, and the ones that are left I can imagine defending, and then I might say “Yesterday I bought some at the store, but I think maybe the last time I ate one (like with a fork and a steakknife and everything) it was about 5-9 days ago at Texas Roadhouse… that was certainly a vivid event because it was my first time back there since covid started” and then just now I became uncertain, and I tried to imagine other events, like what about smaller pieces of steak, and then I remembered some carne asada 3 days ago at a BBQ.
What I think is happening here is that (like the Halle Berry neuron found in the hippocampus of the brain surgery patient) there is at least one steak neuron in my own hippocampus, and it can be stimulated by hearing the word, and persistent firing of it will cause episodic memories (nearly always associated with places) to rise up. Making the activations of cortex-level sensory details and models conform to “the ways that the entire brain can or would be different if the remembered episode was being generated from sensory stimulation (or in this case the echo of that as a memory)”.
So I think hunger representations, mapped through very low level food representations, could push through into episodic memories, and the difference between a memory and a plan is not that large?
Just as many food representing neurons could be stimulated by deficiency detecting sensory neurons, the food ideas would link to food memories, and food memories could become prompts to “go back to that place and try a similar action to what is remembered”.
And all the possible places to go could be activated in parallel in the brain, with winnowing, until a handful of candidates get the most firing because of numerous simultaneous “justifications” that route through numerous memories or variations of action that would all “be good enough”.
The model I have is sort of like… maybe lightning?
An entire cloud solves the problem of finding a very low energy path for electrons to take to go from electron dense places to places that lack electrons, first tentatively and widely, then narrowly and quickly.
Similarly, I suspect the entire brain solves the problem of finding a fast cheap way to cause the muscles to fire in a way that achieves what the brain stem thinks would be desirable, first tentatively and widely, then narrowly and quickly.
I googled [thinking of food fMRI] and found a paper suggesting: hippocampus, insula, caudate.
Then I googled [food insula] and [food caudate] in different tabs. To a first approximation, it looks like the caudate is related to “skilled reaching” for food? Leaving, by process of elimination: the insula?
And uh… yup? The insula seems to keep track of the taste and “goal-worthiness” of foods?
In this review, we will specifically focus on the involvement of the insula in food processing and on multimodal integration of food-related items. Influencing factors of insular activation elicited by various foods range from calorie-content to the internal physiologic state, body mass index or eating behavior. Sensory perception of food-related stimuli including seeing, smelling, and tasting elicits increased activation in the anterior and mid-dorsal part of the insular cortex. Apart from the pure sensory gustatory processing, there is also a strong association with the rewarding/hedonic aspects of food items, which is reflected in higher insular activity and stronger connections to other reward-related areas.
--> hippocampus (also triggerable by active related ideas?) --> memories sifted --> plans (also loop back to hippocampus if plans trigger new memories?) -->
--> prefrontal cortex(?) eventualy STOPS saying “no go” on current best mishmash of a plan -->
--> caudate (and presumably cerebellum) generate --> skilled food seeking firing of muscles to act in imagined way!
The arrows represent sort of “psychic motivational energy” (if we are adopting a theory of mind) as well as “higher firing rate” as well as maybe “leading indicator of WHICH earlier firing predicts WHICH later firing by neurons/activities being pointed to”.
I think you have some theories that there’s quite a few low level subsystems that basically do supervised learning on their restricted domain? My guess is that the insula is where the results of supervised learning on “feeling better after consuming something” are tracked?
Also, it looks like the insula’s supervised learning algorithms can be hacked?
The insular cortex subserves visceral-emotional functions, including taste processing, and is implicated in drug craving and relapse. Here, via optoinhibition, we implicate projections from the anterior insular cortex to the nucleus accumbens as modulating highly compulsive-like food self-administration behaviors
Trying to reconcile this with your “telencephalon” focus… I just learned that the brain has FIVE lobes of the cortex, instead of the FOUR that I had previously thought existed?! At least Encarta used to assert that there are five...
These and other sulci and gyri divide the cerebrum into five lobes: the frontal, parietal, temporal, and occipital lobes and the insula. [bold not in original]
Until I looked up the anatomy, I had just assumed that the insula was part of the brain stem, and so I thought I won some bayes points for my “hard wiring” assumption, but the insula is “the lobe” hiding in the valley between the temporal cortex and the rest of the visible surface lobes, so it is deep down, closer to the brain stem… So maybe you win some bayes points for your telencephalon theory? :-)
there is at least one steak neuron in my own hippocampus, and it can be stimulated by hearing the word, and persistent firing of it will cause episodic memories...to rise up
Oh yeah, I definitely agree that this is an important dynamic. I think there are two cases. In the case of episodic memory I think you’re kinda searching for one of a discrete (albeit large) set of items, based on some aspect of the item. So this is a pure autoassociative memory mechanism. The other case is when you’re forming a brand new thought. I think of it like, your thoughts are made up of a bunch of little puzzle pieces that can snap together, but only in certain ways (e.g. you can’t visualize a “falling stationary rock”, but you can visualize a “blanket made of banana peels”). I think you can issue top-down mandates that there should be a thought containing a certain small set of pieces, and then your brain will search for a way to build out a complete thought (or plan) that includes those pieces. Like “wanting to fit the book in the bag” looks like running a search for a self-consistent thought that ends with the book sliding smoothly into the bag. There might be some autoassociative memory involved here too, not sure, although I think it mainly winds up vaguely similar to belief-propagation algorithms in Bayesian PGMs.
Anyway, the hunger case could look like invoking the piece-of-a-thought:
Piece-of-a-thought X: “[BLANK] and then I eat yummy food”
…and then the search algorithm looks for ways to flesh that out into a complete plausible thought.
I guess your model is more like “the brainstem reaches up and activates Piece-of-a-thought X” and my model is more like “the brainstem waits patiently for the cortex to activate Piece-of-a-thought X, and as soon as it does, it says YES GOOD THANKS, HERE’S SOME REWARD”. And then very early in infancy the cortex learns (by RL) that when its own interoceptive inputs indicate hunger, then it should activate piece-of-a-thought X.
Maybe you’ll say: eating is so basic, this RL mechanism seems wrong. Learning takes time, but infants need to eat, right? But then my response would be: eating is basic and necessary from birth, but doesn’t need to involve the cortex. There can be a hardwired brainstem circuit that says “if you see a prey animal, chase it and kill it”, and another that says “if you smell a certain smell, bite on it”, and another that says “when there’s food in your mouth, chew it and swallow it”, etc. The cortex is for learning more complicated patterns, I think, and by the time it’s capable of doing useful things in general, it can also learn this one simple little pattern, i.e. that hunger signals imply reward-for-thinking-about-eating.
insula
FWIW, in the scheme here, one part of insular cortex is an honorary member of the “agranular prefrontal cortex” club—that’s based purely on this quote I found in Wise 2017: “Although the traditional anatomical literature often treats the orbitofrontal and insular cortex as distinct entities, a detailed analysis of their architectonics, connections, and topology revealed that the agranular insular areas are integral parts of an “orbital prefrontal network””. So this is a “supervised learning” part (if you believe me), and I agree with you that it may well more specifically involve predictions about “feeling better after consuming something”. I also think this is probably the part relevant to your comment “the insula’s supervised learning algorithms can be hacked?”.
Another part of the insula is what Lisa Feldman Barrett calls “primary interoceptive cortex”, i.e. she is suggesting that it learns a vocabulary of patterns that describe incoming interoceptive (body status) signals, analogously to how primary visual cortex learns a vocabulary of patterns that describe incoming visual signals, primary auditory cortex learns a vocabulary of patterns that describe incoming auditory signals, etc.
Those are the two parts of the insula that I know about. There might be other things in the insula too.
caudate
I didn’t explicitly mention caudate here but it’s half of “dorsal striatum”. The other half is putamen—I think they’re properly considered as one structure. “Dorsal striatum” is the striatum associated with motor-control cortex and executive-function cortex, more or less. I’m not sure how that breaks down between caudate and putamen. I’m also not sure why caudate was active in that fMRI paper you found.
hippocampus
I think I draw more of a distinction between plans and memories than you, and put hippocampus on the “memory” side. (I’m thinking roughly “hippocampus = navigation (in all mammals) and first-person memories (only in humans)”, and “dorsolateral prefrontal cortex is executive function and planning (in humans)”.) I’m not sure exactly what the fMRI task was, but maybe it involved invoking memories?
Thanks!! I largely agree with what you wrote.
I was focusing on the implementation of a particular aspect of that. Specifically, when you’re doing what you call “thing modeling”, the “things” you wind up with are entries in a complicated learned world-model—e.g. “thing #6564457” is a certain horrifically complicated statistical regularity in multimodal sensory data, something like: “thing #6564457“ is a prediction that thing #289347 is present, and thing #89672, and thing #68972, but probably not thing #903672”, or whatever.
Meanwhile I agree with you that there is some brainstem / hypothalamus function (outside the learned world-model) that can evaluate how biologically adaptive it would be to eat food with a certain profile of flavors / smells / etc., given the current readings of various sensors detecting nutrient deficiencies etc. (That component actually seems quite straightforward to me.)
And then my particular focus is how exactly the brain builds interface into and out of the world-model, which is a prerequisite for learning that this particular statistical regularity (in the learned world-model) corresponds to a particular vector of sweetness, savoriness, etc. (in the brainstem), which the brainstem can analyze and declare likely to satisfy current physiological needs, and then therefore let’s try to eat it (back in the learned world-model).
If you look closely at what you wrote, I think you’ll find a few places where you need to transfer information into and out of the learned world-model. That’s fine, but there has to be a way that that works, and that’s the part I was especially interested in.
I guess my underlying assumption is that this interfacing isn’t a trivial thing—like I don’t think you can just casually say “The world-model shall have an item type called ‘food’ in it” and then there’s an obvious way to make that happen. I think the world-model is built from the ground up, as learned patterns, and patterns in the patterns, etc., so you generally need stories for how things are learned. At any rate, that type of algorithm seems plausible to me (from everything I’ve read and thought about), and I haven’t seen any alternative story that makes sense to me so far.
It sounds like you want to start with the hypothalamus & brainstem providing a ranked list of all possible foods, and then the world-model finds one that can be eaten. But I want to go basically in the opposite direction, where the planner (working within the learned world-model) proposes a thought that involves eating (“I could eat that carrot in the fridge”), and the hypothalamus & brainstem evaluate how appealing that plan is (“carrots are sweet, I’m low in sugar right now, 7/10″), and then it sends dopamine to reward the thinking of appealing thoughts (and moreso if they’re part of realistic likely-to-succeed plans). Like, if I’m really hungry, I think I’m more likely to decide to eat the first easily-accessible food that pops into my head, rather than working my way through a long list of possible foods that would be hypothetically better (“no I don’t have a coconut smoothie, no I don’t have fried clams, …”). Then, through the magic of reinforcement learning, the planner gradually learns to skillfully and quickly come up with appropriate and viable foods.
Response appreciated! Yeah. I think I have two hunches here that cause me to speak differently.
One of these hunches is that hunger sensors are likely to be very very “low level”, and motivationally “primary”. The other is maybe an expectation that almost literally “every possible thought” is being considered simultaneously in the brain by default, but most do not rise to awareness, or action-production, or verbalizability?
Like I think that hunger sensors firing will cause increased firing in “something or other” that sort of “represents food” (plus giving the food the halo of temporary desirability) and I expect this firing rate to basically go up over time… more hungriness… more food awareness?
Like if you ask me “Jennifer, when was the last time you ate steak?” then I am aware of a wave of candidate answers, and many fall away, and the ones that are left I can imagine defending, and then I might say “Yesterday I bought some at the store, but I think maybe the last time I ate one (like with a fork and a steakknife and everything) it was about 5-9 days ago at Texas Roadhouse… that was certainly a vivid event because it was my first time back there since covid started” and then just now I became uncertain, and I tried to imagine other events, like what about smaller pieces of steak, and then I remembered some carne asada 3 days ago at a BBQ.
What I think is happening here is that (like the Halle Berry neuron found in the hippocampus of the brain surgery patient) there is at least one steak neuron in my own hippocampus, and it can be stimulated by hearing the word, and persistent firing of it will cause episodic memories (nearly always associated with places) to rise up. Making the activations of cortex-level sensory details and models conform to “the ways that the entire brain can or would be different if the remembered episode was being generated from sensory stimulation (or in this case the echo of that as a memory)”.
So I think hunger representations, mapped through very low level food representations, could push through into episodic memories, and the difference between a memory and a plan is not that large?
Just as many food representing neurons could be stimulated by deficiency detecting sensory neurons, the food ideas would link to food memories, and food memories could become prompts to “go back to that place and try a similar action to what is remembered”.
And all the possible places to go could be activated in parallel in the brain, with winnowing, until a handful of candidates get the most firing because of numerous simultaneous “justifications” that route through numerous memories or variations of action that would all “be good enough”.
The model I have is sort of like… maybe lightning?
An entire cloud solves the problem of finding a very low energy path for electrons to take to go from electron dense places to places that lack electrons, first tentatively and widely, then narrowly and quickly.
Similarly, I suspect the entire brain solves the problem of finding a fast cheap way to cause the muscles to fire in a way that achieves what the brain stem thinks would be desirable, first tentatively and widely, then narrowly and quickly.
I googled [thinking of food fMRI] and found a paper suggesting: hippocampus, insula, caudate.
Then I googled [food insula] and [food caudate] in different tabs. To a first approximation, it looks like the caudate is related to “skilled reaching” for food? Leaving, by process of elimination: the insula?
And uh… yup? The insula seems to keep track of the taste and “goal-worthiness” of foods?
So my theory is:
Biochemistry --> hunger sensors (rising, enduring) --> insula (rising, enduring) -->
--> hippocampus (also triggerable by active related ideas?) --> memories sifted --> plans (also loop back to hippocampus if plans trigger new memories?) -->
--> prefrontal cortex(?) eventualy STOPS saying “no go” on current best mishmash of a plan -->
--> caudate (and presumably cerebellum) generate --> skilled food seeking firing of muscles to act in imagined way!
The arrows represent sort of “psychic motivational energy” (if we are adopting a theory of mind) as well as “higher firing rate” as well as maybe “leading indicator of WHICH earlier firing predicts WHICH later firing by neurons/activities being pointed to”.
I think you have some theories that there’s quite a few low level subsystems that basically do supervised learning on their restricted domain? My guess is that the insula is where the results of supervised learning on “feeling better after consuming something” are tracked?
Also, it looks like the insula’s supervised learning algorithms can be hacked?
Trying to reconcile this with your “telencephalon” focus… I just learned that the brain has FIVE lobes of the cortex, instead of the FOUR that I had previously thought existed?! At least Encarta used to assert that there are five...
Until I looked up the anatomy, I had just assumed that the insula was part of the brain stem, and so I thought I won some bayes points for my “hard wiring” assumption, but the insula is “the lobe” hiding in the valley between the temporal cortex and the rest of the visible surface lobes, so it is deep down, closer to the brain stem… So maybe you win some bayes points for your telencephalon theory? :-)
Thanks! This is very interesting!
Oh yeah, I definitely agree that this is an important dynamic. I think there are two cases. In the case of episodic memory I think you’re kinda searching for one of a discrete (albeit large) set of items, based on some aspect of the item. So this is a pure autoassociative memory mechanism. The other case is when you’re forming a brand new thought. I think of it like, your thoughts are made up of a bunch of little puzzle pieces that can snap together, but only in certain ways (e.g. you can’t visualize a “falling stationary rock”, but you can visualize a “blanket made of banana peels”). I think you can issue top-down mandates that there should be a thought containing a certain small set of pieces, and then your brain will search for a way to build out a complete thought (or plan) that includes those pieces. Like “wanting to fit the book in the bag” looks like running a search for a self-consistent thought that ends with the book sliding smoothly into the bag. There might be some autoassociative memory involved here too, not sure, although I think it mainly winds up vaguely similar to belief-propagation algorithms in Bayesian PGMs.
Anyway, the hunger case could look like invoking the piece-of-a-thought:
Piece-of-a-thought X: “[BLANK] and then I eat yummy food”
…and then the search algorithm looks for ways to flesh that out into a complete plausible thought.
I guess your model is more like “the brainstem reaches up and activates Piece-of-a-thought X” and my model is more like “the brainstem waits patiently for the cortex to activate Piece-of-a-thought X, and as soon as it does, it says YES GOOD THANKS, HERE’S SOME REWARD”. And then very early in infancy the cortex learns (by RL) that when its own interoceptive inputs indicate hunger, then it should activate piece-of-a-thought X.
Maybe you’ll say: eating is so basic, this RL mechanism seems wrong. Learning takes time, but infants need to eat, right? But then my response would be: eating is basic and necessary from birth, but doesn’t need to involve the cortex. There can be a hardwired brainstem circuit that says “if you see a prey animal, chase it and kill it”, and another that says “if you smell a certain smell, bite on it”, and another that says “when there’s food in your mouth, chew it and swallow it”, etc. The cortex is for learning more complicated patterns, I think, and by the time it’s capable of doing useful things in general, it can also learn this one simple little pattern, i.e. that hunger signals imply reward-for-thinking-about-eating.
FWIW, in the scheme here, one part of insular cortex is an honorary member of the “agranular prefrontal cortex” club—that’s based purely on this quote I found in Wise 2017: “Although the traditional anatomical literature often treats the orbitofrontal and insular cortex as distinct entities, a detailed analysis of their architectonics, connections, and topology revealed that the agranular insular areas are integral parts of an “orbital prefrontal network””. So this is a “supervised learning” part (if you believe me), and I agree with you that it may well more specifically involve predictions about “feeling better after consuming something”. I also think this is probably the part relevant to your comment “the insula’s supervised learning algorithms can be hacked?”.
Another part of the insula is what Lisa Feldman Barrett calls “primary interoceptive cortex”, i.e. she is suggesting that it learns a vocabulary of patterns that describe incoming interoceptive (body status) signals, analogously to how primary visual cortex learns a vocabulary of patterns that describe incoming visual signals, primary auditory cortex learns a vocabulary of patterns that describe incoming auditory signals, etc.
Those are the two parts of the insula that I know about. There might be other things in the insula too.
I didn’t explicitly mention caudate here but it’s half of “dorsal striatum”. The other half is putamen—I think they’re properly considered as one structure. “Dorsal striatum” is the striatum associated with motor-control cortex and executive-function cortex, more or less. I’m not sure how that breaks down between caudate and putamen. I’m also not sure why caudate was active in that fMRI paper you found.
I think I draw more of a distinction between plans and memories than you, and put hippocampus on the “memory” side. (I’m thinking roughly “hippocampus = navigation (in all mammals) and first-person memories (only in humans)”, and “dorsolateral prefrontal cortex is executive function and planning (in humans)”.) I’m not sure exactly what the fMRI task was, but maybe it involved invoking memories?