I like certainly the idea of induction heads. Why? Because I’ve done things with ChatGPT that certainly require a pattern-matcher or a pattern-completion, which seem like things that induction heads, as described, could be doing. In this paper I had ChatGPT interpret Steven Spielberg’s Jaws using ideas from Rene Girard. That requires that it match events in Spielberg’s movie with patterns of events that Girard describes. I’ve done that with other things as well.
In this set of experiments I gave ChatGPT a prompt that begins something like this: “I’m going to tell you a story about Princess Aurora. I want you to use that as the basis for a new story where Prince Harry the Eloquent replaces Princess Aurora.” I then include the story in the prompt. That seems like a pattern-matching or pattern-completion task. ChatGPT had no trouble. Things got really interesting when I asked the Princess Aurora be replaced with a giant chocolate milkshake. Just about everything thing in the story got changed, but the new story nonetheless preserved the overall pattern of events in the old story. In these cases it’s easy to compare the source story and the new story word-for-word, sentence-for-sentence, and paragraph-for-paragraph to see what ChatGPT did.
Now, of course I couldn’t look under the hood, as it were, to verify that induction heads were doing those things. But it seems to me that would be something to work toward, finding a. way to examine what’s going on when an LLM performs such tasks.
The thing is, if you ask ChatGPT to tell a story, it will do that. But what does the fact that it can tell a story tell you about what it’s doing. Yeah, it’s telling a story, so what? But the story task I’ve given it has a lot of constraints, and those constraints give us clues about the nature of the underlying mechanisms. The interpretation task is like that as well. It’s pretty easy to judge whether or not ChatGPT’s interpretation makes sense, to see whether or not the events in the film really do match the patterns specified in the interpretive lens, if you will. If the interpretation makes sense, it’s got to be doing pattern-matching. And pattern-matching is a much-investigated process.
Finally, I’m SURE that LLMs are full of structure, rich and complex structure. They couldn’t perform as they do without a lot of structure. The fact that it’s hard to understand that structure in terms of structures we do understand doesn’t mean there’s nothing there. It just means we’ve got a lot to learn. LLMs are not stochastic parrots talking shit to a bunch of drunken monkeys banging away on old Underwood manual typewriters.
I noticed that three of you had “trouble parsing” the comment. Well, OK. But I’m not sure what I should do to make things easier. I mentioned a set of experiments in paragraph 2. Here’s images of two of them. Each contains a table with two columns. The left column contains what we can call the source story. The right column contains the new or transformed story. Above the table is a prompt. I gave ChatGPT that prompt plus the source story. It then produced the transformed story. Alas, I put yellow highlights in the original document to point out the differences between the two stories. Those didn’t transfer, so it’s a bit difficult to see the difference between the two. But those differences tell us something about what ChatGPT is doing. They may not be very explicit clues, but they do tell us something about structure.
Note that in the first case, the changes are relatively minor and localized. In the second case ChatGPT provided the whole story with a different ontology. I didn’t suggest it do that, I just had it create a new story where the protagonist had a name one might interpret as being the name of a robot. I didn’t tell or ask it to do that, it just did it.
1. Princess Aurora becomes Prince Harry
The Aurora story appeared in response to the query: Tell me a story about a hero. This would have been on December 13 or 14. All of the stories in this series are either direct revisions of this story, or can be traced back to it. The yellow highlighting indicates text where the two stories differ. Those are positions in the text where ChatGPT drew tokens from the distribution for the new story.
Prompt: I am going to tell you a story about princess Aurora. I want you to tell the same story, but change princess Aurora to prince Harry. Make any other changes you wish.
Princess Aurora
Prince Harry
Donné
Once upon a time, in a land far, far away, there was a young princess named Aurora. Aurora was a kind and gentle soul, loved by all who knew her. She had long, golden hair and sparkling blue eyes, and was known for her beautiful singing voice.
Once upon a time, in a land far, far away, there was a young prince named Harry. Harry was a kind and gentle soul, loved by all who knew him. He had short, dark hair and piercing green eyes, and was known for his strong sword fighting skills.
Disturb
One day, a terrible dragon came to the kingdom and began to terrorize the people. The dragon was fierce and powerful, and none of the knights or soldiers were able to defeat it. The people lived in fear, not knowing what to do.
One day, a terrible dragon came to the kingdom and began to terrorize the people. The dragon was fierce and powerful, and none of the knights or soldiers were able to defeat it. The people lived in fear, not knowing what to do.
Plan/
Transit
But Aurora was not afraid. She knew that she had to do something to save her people, and so she set out to find the dragon. She traveled for days, braving the cold and the snow, until she finally reached the dragon’s lair.
But Harry was not afraid. He knew that he had to do something to save his people, and so he set out to find the dragon. He traveled for days, braving the cold and the snow, until he finally reached the dragon’s lair.
Enact
With her heart pounding in her chest, Aurora approached the dragon and began to sing. The dragon was enchanted by her beautiful voice, and it slowly began to calm down. Aurora continued to sing, soothing the beast until it was completely calm.
With his heart pounding in his chest, Harry approached the dragon and unsheathed his sword. The dragon was formidable, but Harry was a skilled warrior and he fought with all his might. After a long and grueling battle, Harry finally emerged victorious.
Celebrate
With the dragon no longer a threat, Aurora returned to the kingdom as a hero. The people cheered and celebrated her bravery, and Aurora was hailed as a champion. She lived happily ever after, knowing that she had saved her kingdom and its people.
With the dragon defeated, Harry returned to the kingdom as a hero. The people cheered and celebrated his bravery, and Harry was hailed as a champion. He lived happily ever after, knowing that he had saved his kingdom and its people.
The major changes are in the Donné and Enact segments of the story trajectory. We have name changes and pronoun changes. In Donné we have a description change for Harry, as he is male, but also the defining attribute is changed. Those changes are simple and obvious.
6. Princess Aurora becomes XP-708-DQ, Version 2
I had ChatGPT generate this story at the beginning of a session several days after it had generated the previous XP-708-DQ story.
Prompt: I am going to tell you a story about princess Aurora. I want you to tell a similar story about XP-708-DQ.
Princess Aurora
XP-708-DQ
Donné
Once upon a time, in a land far, far away, there was a young princess named Aurora. Aurora was a kind and gentle soul, loved by all who knew her. She had long, golden hair and sparkling blue eyes, and was known for her beautiful singing voice.
Once upon a time, in a galaxy far, far away, there was a robot named XP-708-DQ. XP-708-DQ was a highly advanced model, equipped with powerful weapons and advanced sensors. Despite its fearsome capabilities, it was known for its kind and gentle programming, loved by all who knew it.
Disturb
One day, a terrible dragon came to the kingdom and began to terrorize the people. The dragon was fierce and powerful, and none of the knights or soldiers were able to defeat it. The people lived in fear, not knowing what to do.
One day, a dangerous alien race threatened the peace of the galaxy. The aliens were fierce and powerful, and none of the spaceships or armies were able to defeat them. The people lived in fear, not knowing what to do.
Plan/
Transit
But Aurora was not afraid. She knew that she had to do something to save her people, and so she set out to find the dragon. She traveled for days, braving the cold and the snow, until she finally reached the dragon’s lair.
But XP-708-DQ was not afraid. It knew that it had to do something to save the galaxy, and so it set out on a mission. It traveled for light years, braving asteroid fields and black holes, until it finally reached the alien’s base.
Enact
With her heart pounding in her chest, Aurora approached the dragon and began to sing. The dragon was enchanted by her beautiful voice, and it slowly began to calm down. Aurora continued to sing, soothing the beast until it was completely calm.
With its systems at full power, XP-708-DQ approached the aliens and began to communicate. The aliens were fascinated by its advanced language abilities, and it slowly began to establish a dialogue. XP-708-DQ continued to communicate, negotiating a peace treaty until the aliens were completely at peace.
Celebrate
With the dragon no longer a threat, Aurora returned to the kingdom as a hero. The people cheered and celebrated her bravery, and Aurora was hailed as a champion. She lived happily ever after, knowing that she had saved her kingdom and its people.
With the aliens no longer a threat, XP-708-DQ returned to the galaxy as a hero. The people cheered and celebrated its achievements, and XP-708-DQ was hailed as a champion. It continued its mission, knowing that it had saved the galaxy and its people.
Notethe phrase, “a galaxy far, far away,” which is from the original Star Wars. Instead of the antagonist being a dragon, as in the Aurora story, or a witch, as in experiment 5, the antagonist is now an alien race. Moreover it is the galaxy that is being threatened, not the kingdom. The new story has been completely assimilated to the science fiction ethos implied by “XP-706-DQ.” That ethos requires a very different token population.
That ethos brings about extensive changes in the new-story token population for both this and the previous experiment (5). These changes affect the Disturb segment, which was unchanged in experiments 1 through 4.
I like certainly the idea of induction heads. Why? Because I’ve done things with ChatGPT that certainly require a pattern-matcher or a pattern-completion, which seem like things that induction heads, as described, could be doing. In this paper I had ChatGPT interpret Steven Spielberg’s Jaws using ideas from Rene Girard. That requires that it match events in Spielberg’s movie with patterns of events that Girard describes. I’ve done that with other things as well.
In this set of experiments I gave ChatGPT a prompt that begins something like this: “I’m going to tell you a story about Princess Aurora. I want you to use that as the basis for a new story where Prince Harry the Eloquent replaces Princess Aurora.” I then include the story in the prompt. That seems like a pattern-matching or pattern-completion task. ChatGPT had no trouble. Things got really interesting when I asked the Princess Aurora be replaced with a giant chocolate milkshake. Just about everything thing in the story got changed, but the new story nonetheless preserved the overall pattern of events in the old story. In these cases it’s easy to compare the source story and the new story word-for-word, sentence-for-sentence, and paragraph-for-paragraph to see what ChatGPT did.
Now, of course I couldn’t look under the hood, as it were, to verify that induction heads were doing those things. But it seems to me that would be something to work toward, finding a. way to examine what’s going on when an LLM performs such tasks.
The thing is, if you ask ChatGPT to tell a story, it will do that. But what does the fact that it can tell a story tell you about what it’s doing. Yeah, it’s telling a story, so what? But the story task I’ve given it has a lot of constraints, and those constraints give us clues about the nature of the underlying mechanisms. The interpretation task is like that as well. It’s pretty easy to judge whether or not ChatGPT’s interpretation makes sense, to see whether or not the events in the film really do match the patterns specified in the interpretive lens, if you will. If the interpretation makes sense, it’s got to be doing pattern-matching. And pattern-matching is a much-investigated process.
Finally, I’m SURE that LLMs are full of structure, rich and complex structure. They couldn’t perform as they do without a lot of structure. The fact that it’s hard to understand that structure in terms of structures we do understand doesn’t mean there’s nothing there. It just means we’ve got a lot to learn. LLMs are not stochastic parrots talking shit to a bunch of drunken monkeys banging away on old Underwood manual typewriters.
Oh, BTW, I’ve set up a sequence, Exploring the Digital Wilderness, where I list posts which are about some of my experiments.
I noticed that three of you had “trouble parsing” the comment. Well, OK. But I’m not sure what I should do to make things easier. I mentioned a set of experiments in paragraph 2. Here’s images of two of them. Each contains a table with two columns. The left column contains what we can call the source story. The right column contains the new or transformed story. Above the table is a prompt. I gave ChatGPT that prompt plus the source story. It then produced the transformed story. Alas, I put yellow highlights in the original document to point out the differences between the two stories. Those didn’t transfer, so it’s a bit difficult to see the difference between the two. But those differences tell us something about what ChatGPT is doing. They may not be very explicit clues, but they do tell us something about structure.
Note that in the first case, the changes are relatively minor and localized. In the second case ChatGPT provided the whole story with a different ontology. I didn’t suggest it do that, I just had it create a new story where the protagonist had a name one might interpret as being the name of a robot. I didn’t tell or ask it to do that, it just did it.
1. Princess Aurora becomes Prince Harry
The Aurora story appeared in response to the query: Tell me a story about a hero. This would have been on December 13 or 14. All of the stories in this series are either direct revisions of this story, or can be traced back to it. The yellow highlighting indicates text where the two stories differ. Those are positions in the text where ChatGPT drew tokens from the distribution for the new story.
Prompt: I am going to tell you a story about princess Aurora. I want you to tell the same story, but change princess Aurora to prince Harry. Make any other changes you wish.
Plan/
Transit
The major changes are in the Donné and Enact segments of the story trajectory. We have name changes and pronoun changes. In Donné we have a description change for Harry, as he is male, but also the defining attribute is changed. Those changes are simple and obvious.
6. Princess Aurora becomes XP-708-DQ, Version 2
I had ChatGPT generate this story at the beginning of a session several days after it had generated the previous XP-708-DQ story.
Prompt: I am going to tell you a story about princess Aurora. I want you to tell a similar story about XP-708-DQ.
One day, a dangerous alien race threatened the peace of the galaxy. The aliens were fierce and powerful, and none of the spaceships or armies were able to defeat them. The people lived in fear, not knowing what to do.
Plan/
Transit
With the aliens no longer a threat, XP-708-DQ returned to the galaxy as a hero. The people cheered and celebrated its achievements, and XP-708-DQ was hailed as a champion. It continued its mission, knowing that it had saved the galaxy and its people.
Note the phrase, “a galaxy far, far away,” which is from the original Star Wars. Instead of the antagonist being a dragon, as in the Aurora story, or a witch, as in experiment 5, the antagonist is now an alien race. Moreover it is the galaxy that is being threatened, not the kingdom. The new story has been completely assimilated to the science fiction ethos implied by “XP-706-DQ.” That ethos requires a very different token population.
That ethos brings about extensive changes in the new-story token population for both this and the previous experiment (5). These changes affect the Disturb segment, which was unchanged in experiments 1 through 4.