This post is great, and I strong-upvoted it. But I was left wishing that some of the more evocative mathematical phrases (“the waluigi eigen-simulacra are attractor states of the LLM”) could really be grounded into a solid mechanistic theory that would make precise, testable predictions. But perhaps such a yearning on the part of the reader is the best possible outcome of the post.
I did consider avoiding technical mathematical terminology because it would suggest a level of mathematical rigour that doesn’t actually exist. But I decided to keep the mathematical terminology but hope that people interpret it loosely.
I really enjoyed the absurdity of mathematical terms in close proximity to Super Mario characters. It was simultaneously enlightening and humorous. I found the simulacra superposition concept in particular to be a useful framing.
In addition to “The Waluigi eigen-simulacra are attractor states of the LLM”, the following bit provided valuable insight while making me chuckle at the sheer geekiness:
“However, the superposition is unlikely to collapse to the Luigi simulacrum [...] This is formally connected to the asymmetry of the Kullback-Leibler divergence.”
Literary Mode: Literary experience is mediated by a mode of neural activity in which one’s primary attention is removed form the external world and invested in the text. The properties of literary works are fitted to that mode of activity.
Extralinguistic Grounding: Literary language is linked to extralinguistic sensory and motor schemas in a way that is essential to literary experience.
Form: The form of a given work can be said to be a computational structure.
Sharability: That computational form is the same for all competent readers.
Character as Computational Unit: Individual characters can be treated as unified computational units in some, but not necessarily all, literary forms.
Armature Invariance: The relationships between the entities in the armature of a literary work are the same for all readers.
Elasticity: The meaning of literary works is elastic and can readily accommodate differences in expressive detail and differences among individuals.
Increasing Formal Sophistication: The long-term course of literary history has been toward forms of increasing sophistication.
Ranks: Over the long-term literary history has so far evolved forms at four successive cognitive ranks. These are correlated with a richer and more flexible construction of the self.
This post is great, and I strong-upvoted it. But I was left wishing that some of the more evocative mathematical phrases (“the waluigi eigen-simulacra are attractor states of the LLM”) could really be grounded into a solid mechanistic theory that would make precise, testable predictions. But perhaps such a yearning on the part of the reader is the best possible outcome of the post.
Thanks for the kind words.
I did consider avoiding technical mathematical terminology because it would suggest a level of mathematical rigour that doesn’t actually exist. But I decided to keep the mathematical terminology but hope that people interpret it loosely.
I really enjoyed the absurdity of mathematical terms in close proximity to Super Mario characters. It was simultaneously enlightening and humorous. I found the simulacra superposition concept in particular to be a useful framing.
In addition to “The Waluigi eigen-simulacra are attractor states of the LLM”, the following bit provided valuable insight while making me chuckle at the sheer geekiness:
“However, the superposition is unlikely to collapse to the Luigi simulacrum [...] This is formally connected to the asymmetry of the Kullback-Leibler divergence.”
Welcome to literary theory in the 21st century.
any thoughts about how to ground them?
I will have some thoughts in a bit but I am currently busy, just dropping this comment before I can come back and read this properly
It does seem like this post is successfully working towards a mathematical model of narrative structure, with LLMs as a test bed.
YES!
Since structuralist narratology is on the table, you might what to check out what Lévi-Strauss did in The Raw and the Cooked, where he was inspired by algebraic group theory. I discuss that in a working paper: Beyond Lévi-Strauss on Myth: Objectification, Computation, and Cognition, where I also discuss the work Margaret Masterman did on haiku in the Ancient Days. There was a lot of work on story grammars in the 1980s or so and some of that is continuing, especially in the video games world. I have proposed: Literary Morphology: Nine Propositions in a Naturalist Theory of Form (Version 4). The propositions:
Literary Mode: Literary experience is mediated by a mode of neural activity in which one’s primary attention is removed form the external world and invested in the text. The properties of literary works are fitted to that mode of activity.
Extralinguistic Grounding: Literary language is linked to extralinguistic sensory and motor schemas in a way that is essential to literary experience.
Form: The form of a given work can be said to be a computational structure.
Sharability: That computational form is the same for all competent readers.
Character as Computational Unit: Individual characters can be treated as unified computational units in some, but not necessarily all, literary forms.
Armature Invariance: The relationships between the entities in the armature of a literary work are the same for all readers.
Elasticity: The meaning of literary works is elastic and can readily accommodate differences in expressive detail and differences among individuals.
Increasing Formal Sophistication: The long-term course of literary history has been toward forms of increasing sophistication.
Ranks: Over the long-term literary history has so far evolved forms at four successive cognitive ranks. These are correlated with a richer and more flexible construction of the self.