I was slightly surprised to find that even fine-tuning GPT-Neo-125M for a long time on many sequences of letters followed by spaces, followed by a colon, followed by the same sequence in reverse, was not enough to get it to pick up the pattern—probably because the positional encoding vectors make the difference between e.g. “18 tokens away” and “19 tokens away” a rather subtle difference. However, I then tried fine-tuning on a similar dataset with numbers in between (e.g. “1 W 2 O 3 R 4 D 5 S : 5 S 4 D 3 R 2 O 1 W”) (or similar representation—can’t remember exactly, but something roughly like that) and it picked up the pattern right away. Data representation matters a lot!
I tried to use that approach to teach GPT-3 to solve the problem at the top of this post. As you can see, it kinda worked; GPT-3 grasps that some things need to be reversed, but it then goes a bit off the rails (adding a random “this is a great” to the end of my prompt, with the whole phrase reversed rather than each word; then it starts out reversing the individual words of the sentence, but ends up just completing the sentence instead, using the other common completion—“falls” rather than “stays”. Then when it tries to reverse each individual word, it fails completely, and just reorders/reworks the words a bit).
This approach is a little surprising. I would have thought that adding on numbers to my space-separating approach, and then merging space-separated letters into a final solid word, would have tripped up GPT-3 and inevitably led to errors. But, at least with InstructGPT, it works.
I was slightly surprised to find that even fine-tuning GPT-Neo-125M for a long time on many sequences of letters followed by spaces, followed by a colon, followed by the same sequence in reverse, was not enough to get it to pick up the pattern—probably because the positional encoding vectors make the difference between e.g. “18 tokens away” and “19 tokens away” a rather subtle difference. However, I then tried fine-tuning on a similar dataset with numbers in between (e.g. “1 W 2 O 3 R 4 D 5 S : 5 S 4 D 3 R 2 O 1 W”) (or similar representation—can’t remember exactly, but something roughly like that) and it picked up the pattern right away. Data representation matters a lot!
Here’s an example of someone prompting with a walkthrough of a similar token-aware approach to successfully guide GPT-3:
https://twitter.com/npew/status/1525900849888866307
I tried to use that approach to teach GPT-3 to solve the problem at the top of this post. As you can see, it kinda worked; GPT-3 grasps that some things need to be reversed, but it then goes a bit off the rails (adding a random “this is a great” to the end of my prompt, with the whole phrase reversed rather than each word; then it starts out reversing the individual words of the sentence, but ends up just completing the sentence instead, using the other common completion—“falls” rather than “stays”. Then when it tries to reverse each individual word, it fails completely, and just reorders/reworks the words a bit).
Fascinating. Thanks!
This approach is a little surprising. I would have thought that adding on numbers to my space-separating approach, and then merging space-separated letters into a final solid word, would have tripped up GPT-3 and inevitably led to errors. But, at least with InstructGPT, it works.
Thanks; very interesting result.
Fascinating! Thanks for sharing!