The main worry that I have with regards to your approach is how competitive SociaLLM would be with regards to SOTA foundation models given both (1) the different architecture you plan to use, and (2) practical constraints on collecting the requisite structured data. While it is certainly interesting that your architecture lends itself nicely to inducing self-other overlap, if it is not likely to be competitive at the frontier, then the methods uniquely designed to induce self-other overlap on SociaLLM are likely to not scale/transfer well to frontier models that do pose existential risks. (Proactively ensuring transferability is the reason we focus on an additional training objective and make minimal assumptions about the architecture in the self-other overlap agenda.)
I agree with worries (1) and (2). I think there is a way to de-risk this.
The block hierarchy that is responsible for tracking the local context consists of classic Transformer blocks. Only the user’s own history tracking really needs to be an SSM hierarchy because it quickly surpasses the scalability limits of self-attention (also, interlocutor’s tracking blocks in private 1-1 or small group chats that can be arbitrarily long, but there is probably no such data available for training). In the public data (such as forums, public chats rooms, Diplomacy and other text games), the interlocutor’s history traces would 99% of the time easily fit into 100k symbols, but for the symmetry with user’s own state (same weights!) and for having the same representation structure it should mirror the user’s own SSM blocks, of course.
With such an approach, the SSM hierarchies could start very small, with only a few blocks or even just a single SSM block (i.e., two blocks in total: one for user’s own and one for interlocutor’s state), and attach to the middle of the Transformer hierarchy to select from it. However, I think this approach couldn’t be just slapped on the tre-trained LLama or another large Transformer LLM model. I suspect the transformer should be co-trained with the SSM blocks to induce the Transformer to make the corresponding representations useful for the SSM blocks. “Pretraining Language Models with Human Preferences” is my intuition pump here.
Regarding the sufficiency and quality of training data, the Transformer hierarchy itself could still be trained on arbitrary texts, as well as the current LLMs. And we can adjust the size of the SSM hierarchies to the amounts of high-quality dialogue and forum data that we are able to obtain. I think this a no-brainer that this design would improve the frontier quality in LLM apps that value personalisation and attunement to the user’s current state (psychological, emotional, levels of knowledge, etc.), relative to whatever “base” Transformer model we would take (such as Llama, or any other).
One additional worry is that many of the research benefits of SociaLLM may not be out of reach for current foundation models, and so it is unclear if investing in the unique data and architecture setup is worth it in comparison to the counterfactual of just scaling up current methods.
With this I disagree, I think it’s critical for the user state tracking to be energy-based. I don’t think there are ways to recapitulate this with auto-regressive Transformer language models (cf. any LeCun’s presentation from the last year). There are potential ways to recapitulate this with other language modelling architectures (non-Transformer and non-SSM), but they currently don’t hold any stronger promise than SSM, so I don’t see any reasons to pick them.
In another thread, Marc Carauleanu wrote:
I agree with worries (1) and (2). I think there is a way to de-risk this.
The block hierarchy that is responsible for tracking the local context consists of classic Transformer blocks. Only the user’s own history tracking really needs to be an SSM hierarchy because it quickly surpasses the scalability limits of self-attention (also, interlocutor’s tracking blocks in private 1-1 or small group chats that can be arbitrarily long, but there is probably no such data available for training). In the public data (such as forums, public chats rooms, Diplomacy and other text games), the interlocutor’s history traces would 99% of the time easily fit into 100k symbols, but for the symmetry with user’s own state (same weights!) and for having the same representation structure it should mirror the user’s own SSM blocks, of course.
With such an approach, the SSM hierarchies could start very small, with only a few blocks or even just a single SSM block (i.e., two blocks in total: one for user’s own and one for interlocutor’s state), and attach to the middle of the Transformer hierarchy to select from it. However, I think this approach couldn’t be just slapped on the tre-trained LLama or another large Transformer LLM model. I suspect the transformer should be co-trained with the SSM blocks to induce the Transformer to make the corresponding representations useful for the SSM blocks. “Pretraining Language Models with Human Preferences” is my intuition pump here.
Regarding the sufficiency and quality of training data, the Transformer hierarchy itself could still be trained on arbitrary texts, as well as the current LLMs. And we can adjust the size of the SSM hierarchies to the amounts of high-quality dialogue and forum data that we are able to obtain. I think this a no-brainer that this design would improve the frontier quality in LLM apps that value personalisation and attunement to the user’s current state (psychological, emotional, levels of knowledge, etc.), relative to whatever “base” Transformer model we would take (such as Llama, or any other).
With this I disagree, I think it’s critical for the user state tracking to be energy-based. I don’t think there are ways to recapitulate this with auto-regressive Transformer language models (cf. any LeCun’s presentation from the last year). There are potential ways to recapitulate this with other language modelling architectures (non-Transformer and non-SSM), but they currently don’t hold any stronger promise than SSM, so I don’t see any reasons to pick them.