Reviews higher-order theories (HOT) of consciousness and their relation to global workspace theories (GWT) of consciousness, suggesting that HOT and GWT are complementary. Consciousness and the Brain, of course, is a GWT theory; whereas HOT theories suggest that some higher-order representation is (also) necessary for us to be conscious of something. I read the HOT models as being closely connected to introspective awareness; e.g. the authors suggest a connection between alexityhmia (unawareness of your emotions) and abnormalities in brain regions related to higher-order representation.
While the HOT theories seem to suggest that you need higher-order representation of something to be conscious of a thing, I would say that you need higher-order representation of something in order to be conscious of having been conscious of something. (Whether being conscious of something without being conscious of being conscious of it can count as being conscious of it, is of course an interesting philosophical question.)
I have suggested that control of thought and control of behavior operate on similar principles; this paper argues the same.
We often describe our mental states through analogy to physical actions. We hold something in mind or push it out of our thoughts. An emerging question in cognitive control is whether this relationship runs deeper than metaphor, with similar cognitive architectures underpinning our ability to control our physical actions and our mental states. For instance, recent work has shown that analogous control processes serve to optimize performance and regulate brain dynamics for both motor and cognitive actions [1,2]. A new study by Egger and colleagues [3] provides important new clues that the mechanisms supporting motor and cognitive control are more similar than previously shown.
These researchers tested whether the control of internal states exhibits a signature property of the motor system: the reliance on an internal model to guide adjustments of control [4]. To control one’s actions, a person needs to maintain an internal model of their environment (e.g., potential changes in terrain or atmosphere) and of their own motor system (e.g., how successful they are at executing a motor command [5]). This model can be used to generate online predictions about the outcome of an action and to course—correct when there is a mismatch between that prediction and the actual outcome. This process is thought to be implemented via interactions between: (i) a simulator that makes predictions, (ii) an estimator that learns the current state, and (iii) a controller that implements actions. This new study investigated whether neural activity during the control of cognitive processes reflected this same three-part architecture.
To answer this question, Egger and colleagues recorded neural activity while monkeys performed an interval reproduction task (Figure 1). The monkeys observed two samples of a time interval and then timed a saccade to reproduce this interval. Previous work has shown that population-level neural activity in the dorsomedial frontal cortex (DMFC) during similar tasks systematically scales with the timing of an action [6]. If action timing in this task depends on an internal model, then this temporal scaling should already be present in DMFC activity prior to receiving a cue to respond. If the monkeys were not relying on an internal model, and the activity instead reflected the passive measurement of time (‘open-loop’ control), then DMFC activity during the second interval should not exhibit such temporal scaling.
The monkeys’ behavior and neural activity demonstrated that they combined prior knowledge about the average interval duration with their perception of the current interval duration [7]. This behavior was well-captured by a nearoptimal Bayesian algorithm that updated predictions in a way that was biased towards the average interval. By independently varying the duration of the two sample intervals, the authors were further able to show that the monkeys incorporated both samples into their duration estimate.
Signatures of this biased updating process were also observed in DMFC neural activity. Replicating previous studies, individual neurons in the DMFC demonstrated ramping activity during the reproduction of an interval, with faster ramping when the monkey reproduced shorter intervals [6]. Critically, neural activity during the second sample interval exhibited the predicted simulation profile: neurons demonstrated interval-dependent ramping during this epoch, prior to the response cue.
Further support for an internal model hypothesis was found across different measures of neural activity, and in their relationship with subsequent behavior. Temporal scaling was evident not only at the level of DMFC single neurons but also in the population-level neural dynamics across this region. Unlike the transient single-unit responses, the rate of change in these population dynamics scaled consistently with interval length throughout the second sample interval. These dynamics reflected the same Bayesian biases observed in monkeys’ behavior: an initial bias towards the average interval duration that became less biased with more samples. Critically, these population dynamics also predicted when the monkey would saccade on the upcoming response interval, and did so above and beyond what would be predicted by the lengths of the sampled time intervals alone. Collectively, these findings are consistent with the DMFC implementing an internal model to optimize the learning of task goals and the control of neural population dynamics.
This study provides evidence that DMFC mediates the influence of prior predictions and incoming sensory evidence on planned actions, and lays the groundwork for critical tests of this proposed mechanism using causal manipulations (i.e., stimulation or inactivation). Such causal tests can also help to rule out alternative accounts of neural dynamics during the sample intervals, for instance, whether they reflect a simulated motor plan (as the authors infer) or an interval expectation (e.g., predicting the onset of the interval cue [8]). Nevertheless, by elaborating on the neuronal dynamics within DMFC during a task that requires online adjustments of learning and control, this study builds on a growing literature that implicates regions along this dorsomedial wall in the control of motor and cognitive commands [9,10].
More generally, this research provides compelling new evidence that motor and cognitive control share a common computational toolbox. Past work has suggested that both forms of control serve similar objectives (achieving a goal state within a dynamic, uncertain, and noisy environment) and that they are also both constrained by some underlying cost, limiting the amount of control that individuals can engage at a given time. As a consequence, decisions about how to allocate one’s control are sensitive to whether the reward for goal achievement outweighs these costs [10]. To the extent computational and neural architecture for motor and cognitive control allocation mirror one another, the behavior and neural dynamics observed in the current task should demonstrate sensitivity to performance incentives for both forms of control.
In spite of their abundant bodies of research, the obstacle to bridging our understanding of motor and cognitive control have been similarly abundant, including limitations of tasks, measurement tools, and model organisms. This study demonstrates how a combination of computational modeling and measures of neural dynamics in the monkey can be leveraged towards this goal and, in doing so, provides a valuable path forward in mapping the joints between these two domains of control.
In Book summary: Unlocking the Emotional Brain and Building up to an Internal Family Systems model, I referenced models under which a particular event in a person’s life gives rise to a generalized belief schema, and situations which re-activate that belief schema may also partially re-activate recollection of the original event, and vice versa; if something reminds you of a situation you experienced as a child, you may also to some extent reason in the kinds of terms that you did when you were a child and in that situation. This paper discusses connections between episodic memories (e.g., “I remember reading 1984 in Hyde Park yesterday”) and semantic memories (e.g. “1984 was written by George Orwell”), and how activation of one may activate another.
What underlies the overlap between the semantic and recollection networks? We propose that the answer lies in the fact that the content of an episodic memory typically comprises a conjunction of familiar concepts and episode-specific information (such as sensory and spatial context), much as the episodic interpretation of concept cells suggests. Thus, recollection of a prior episode entails the reinstatement not only of contextual information unique to the episode, but also of the conceptual processing that was engaged when the recollected event was experienced (see also [66]). From this perspective, ‘recollection success effects’ in cortical members of the core recollection network do not reflect processing that supports episodic memory per se, but rather, the reinstatement of the conceptual processing that invariably underpins our interactions with the world in real-time (e.g., [10,67,68]). [...]
Although the proposal that recollection success effects in the core network reflect the reinstatement of conceptual processing is both parsimonious and, we contend, consistent with the available evidence, it lacks direct support. fMRI studies examining the neural correlates of successful recollection have invariably used meaningful experimental items, such as concrete words, or pictures of objects, and have typically done so in the context of study tasks that require or encourage semantic elaboration. To our knowledge, with the exception of [89], there are no published studies in which recollection effects were contrasted according to the amount of semantic or conceptual processing engaged during encoding (although see [90] for a study in which encoding was manipulated but the subsequent memory test did not allow identification of items recognized on the basis of recollection rather than on familiarity). In [89], the memory test required a discrimination between unstudied items and items subjected to semantic or nonsemantic study. Retrieval effects in the core network were not fully explored, but intriguingly, one member of the network (left parahippocampal cortex) was reported to demonstrate a greater recollection effect (operationalized as greater activity for correct than incorrect source judgments) for semantically than nonsemantically studied items. This finding is consistent with the present proposal, but it remains to be established whether, as predicted by the proposal, recollection-related activity within the core network as a whole covaries with the amount of semantic processing accorded a recollected episode when it was first experienced. [...]
Thus far, we have discussed episodic and semantic memories without reference to the possibility that their content and neural underpinnings might vary over time. However, there is a long-standing literature documenting that memory representations can be highly dynamic, shifting their dependence from the hippocampus and adjacent regions of the medial temporal lobe (MTL) to other neocortical regions, a phenomenon often referred to as ‘systems consolidation’ [64,65,91–93]. In recent years, systems consolidation has become increasingly intertwined with the construct of memory ‘semanticization’ and schematization, processes by which semantic knowledge and schemas [83] emerge from episodic memory or assimilate aspects of it.
Early studies and theories of memory consolidation, beginning with Ribot and reiterated for almost a century, typically did not distinguish between episodic and semantic memory [65,94–96]. Among the first to realize the importance of the episodic–semantic distinction for theories of memory consolidation were Kinsbourne and Wood [97]. They proposed that traumatic amnesia affected only episodic memory, regardless of the age of the memory, and left semantic and schematic memory relatively preserved. Cases in which remote episodic memories appeared to be preserved were attributed to semanticization or schematization through repeated re-encoding (see remote memory), allowing them to achieve the status of personal facts [98,99].
In an important development of the ‘standard’ model of consolidation, McClelland et al. proposed that the hippocampus maintains episodic representations of an event while communicating with (‘instructing’) the neocortical system to incorporate information about the event into its knowledge structure [100]. It was argued that, to protect the cortical network from catastrophic interference, learning had to be slow, thus providing a principled explanation for the extended time period that systems consolidation was assumed to take. Of importance, the model proposes that, in the process of incorporating an episodic memory into a semantic network, the episodic component, initially dependent on the hippocampus, is lost. This represents an important point of divergence from the standard model, in which episodic information is retained in the neocortex along with semantic information (see later).
Incorporating the original idea of Kinsbourne and Wood [97] and the complementary learning perspective [100], ‘multiple trace theory’ (MTT) [101] proposed that the hippocampus supports episodic memories for as long as they exist. By contrast, the theory proposed that semantic memories depend upon the neocortex, which extracts statistical regularities across distinct episodes. Thus, hippocampal damage should have a profound effect on retention and retrieval of episodic memories of any vintage, while leaving semanticized and schematized memories relatively intact.
While receiving empirical support [64,102] (see also [65,103,104] for examples of convergent findings from studies of experimental animals), MTT has also been subjected to several critiques (e.g., [93,105–108]). However, the essence of the theory resonates with the recurring theme of the present review that episodic and semantic memory are intertwined, yet retain a measure of functional and neural distinctiveness. Since its inception, MTT has been extended [65,104,109] to propose that episodic memories can become transformed to more semantic or schematic versions with time and experience (see ‘Episodic and Semantic Memory in Neurodegenerative Disorders’ section); indeed, in some cases, both the original and the semanticized or schematic version of a memory coexist and engage in dynamic interaction with one another. According to this Trace Transformation Theory, the specific neocortical regions supporting transformed memories differ depending on the kind of information that is retained and retrieved. Correspondingly, for complex events, the transformed memories might depend either on event schemas, or on the gist of the event [110–113]. Increased activation of the vmPFC, believed to be implicated in processing schemas [83], and decreased hippocampal activation have both been reported as details are lost and memories become more gist-like and schematic [83,102,110,113], particularly for memories that are congruent with existing schemas [114,115]. Even when details of remote memories are retained, along with continuing hippocampal activation, there is increased vmPFC activation over time [116,117]. Which memory of an event (e.g., its semanticized or schematic version or the detailed episodic memory of the original event) predominates at retrieval will depend on a variety of factors, such as contextual factors and processing demands (see ‘Semantic memory: Neural Underpinnings’ and ‘Episodic Memory: Neural Underpinnings’ sections), in addition to the availability of one or the other type of information (see also [118]). Thus, retrieval of complex memories depends on the coordinated activation of different combinations of regions (‘process-specific assemblies’ [64,119,120]) belonging to neural networks underlying episodic and semantic memory.
The neuroimaging evidence reviewed to date strongly suggests that successful recollection necessitates the reinstatement not only of sensory-perceptual contextual information characteristic of the original experience, but also the semantic representations and conceptual processing that occurred during that experience. Rather than viewing episodic and semantic memory as dichotomous or mutually exclusive entities, the marked neural overlap between these forms of memory suggests that we must move towards considering the dynamic interplay of sensory-perceptual and conceptual elements during reinstatement of a recollected experience. One way in which we could test this proposal is to examine how progressive neural insult of key structures implicated in episodic and semantic memory impacts related putative functions, including event recollection and event construction.
Recent papers relevant to earlier posts in my multiagent sequence:
Understanding the Higher-Order Approach to Consciousness. Richard Brown, Hakwan Lau, Joseph E.LeDoux. Trends in Cognitive Sciences, Volume 23, Issue 9, September 2019, Pages 754-768.
Reviews higher-order theories (HOT) of consciousness and their relation to global workspace theories (GWT) of consciousness, suggesting that HOT and GWT are complementary. Consciousness and the Brain, of course, is a GWT theory; whereas HOT theories suggest that some higher-order representation is (also) necessary for us to be conscious of something. I read the HOT models as being closely connected to introspective awareness; e.g. the authors suggest a connection between alexityhmia (unawareness of your emotions) and abnormalities in brain regions related to higher-order representation.
While the HOT theories seem to suggest that you need higher-order representation of something to be conscious of a thing, I would say that you need higher-order representation of something in order to be conscious of having been conscious of something. (Whether being conscious of something without being conscious of being conscious of it can count as being conscious of it, is of course an interesting philosophical question.)
Bridging Motor and Cognitive Control: It’s About Time! Harrison Ritz, Romy Frömer, Amitai Shenhav. Trends in Cognitive Sciences, in press.
I have suggested that control of thought and control of behavior operate on similar principles; this paper argues the same.
From Knowing to Remembering: The Semantic–Episodic Distinction. Louis Renoult, Muireann Irish, Morris Moscovitch, and Michael D. Rugg. Trends in Cognitive Sciences, in press.
In Book summary: Unlocking the Emotional Brain and Building up to an Internal Family Systems model, I referenced models under which a particular event in a person’s life gives rise to a generalized belief schema, and situations which re-activate that belief schema may also partially re-activate recollection of the original event, and vice versa; if something reminds you of a situation you experienced as a child, you may also to some extent reason in the kinds of terms that you did when you were a child and in that situation. This paper discusses connections between episodic memories (e.g., “I remember reading 1984 in Hyde Park yesterday”) and semantic memories (e.g. “1984 was written by George Orwell”), and how activation of one may activate another.