Cool experiment! I could imagine that the tokenizer handicaps GPT’s performance here (reversing the characters leads to completely different tokens). With a character-level tokenizer GPT should/might be able to handle that task better!
I was slightly surprised to find that even fine-tuning GPT-Neo-125M for a long time on many sequences of letters followed by spaces, followed by a colon, followed by the same sequence in reverse, was not enough to get it to pick up the pattern—probably because the positional encoding vectors make the difference between e.g. “18 tokens away” and “19 tokens away” a rather subtle difference. However, I then tried fine-tuning on a similar dataset with numbers in between (e.g. “1 W 2 O 3 R 4 D 5 S : 5 S 4 D 3 R 2 O 1 W”) (or similar representation—can’t remember exactly, but something roughly like that) and it picked up the pattern right away. Data representation matters a lot!
I tried to use that approach to teach GPT-3 to solve the problem at the top of this post. As you can see, it kinda worked; GPT-3 grasps that some things need to be reversed, but it then goes a bit off the rails (adding a random “this is a great” to the end of my prompt, with the whole phrase reversed rather than each word; then it starts out reversing the individual words of the sentence, but ends up just completing the sentence instead, using the other common completion—“falls” rather than “stays”. Then when it tries to reverse each individual word, it fails completely, and just reorders/reworks the words a bit).
This approach is a little surprising. I would have thought that adding on numbers to my space-separating approach, and then merging space-separated letters into a final solid word, would have tripped up GPT-3 and inevitably led to errors. But, at least with InstructGPT, it works.
For the similar anagram task, I found space-separating (to avoid the BPE inconsistency/nondeterminism by forcing it to encode individual letters) seemed like it helped: https://gwern.net/GPT-3-nonfiction#anagrams
For this task, I think a worthwhile followup would be to experiment with the new edit mode.
Possibly! Though it did seem to recognise that the words were spelt backwards. It must have some backwards spelt words in its training data, just not that many.
Cool experiment! I could imagine that the tokenizer handicaps GPT’s performance here (reversing the characters leads to completely different tokens). With a character-level tokenizer GPT should/might be able to handle that task better!
I was slightly surprised to find that even fine-tuning GPT-Neo-125M for a long time on many sequences of letters followed by spaces, followed by a colon, followed by the same sequence in reverse, was not enough to get it to pick up the pattern—probably because the positional encoding vectors make the difference between e.g. “18 tokens away” and “19 tokens away” a rather subtle difference. However, I then tried fine-tuning on a similar dataset with numbers in between (e.g. “1 W 2 O 3 R 4 D 5 S : 5 S 4 D 3 R 2 O 1 W”) (or similar representation—can’t remember exactly, but something roughly like that) and it picked up the pattern right away. Data representation matters a lot!
Here’s an example of someone prompting with a walkthrough of a similar token-aware approach to successfully guide GPT-3:
https://twitter.com/npew/status/1525900849888866307
I tried to use that approach to teach GPT-3 to solve the problem at the top of this post. As you can see, it kinda worked; GPT-3 grasps that some things need to be reversed, but it then goes a bit off the rails (adding a random “this is a great” to the end of my prompt, with the whole phrase reversed rather than each word; then it starts out reversing the individual words of the sentence, but ends up just completing the sentence instead, using the other common completion—“falls” rather than “stays”. Then when it tries to reverse each individual word, it fails completely, and just reorders/reworks the words a bit).
Fascinating. Thanks!
This approach is a little surprising. I would have thought that adding on numbers to my space-separating approach, and then merging space-separated letters into a final solid word, would have tripped up GPT-3 and inevitably led to errors. But, at least with InstructGPT, it works.
Thanks; very interesting result.
Fascinating! Thanks for sharing!
For the similar anagram task, I found space-separating (to avoid the BPE inconsistency/nondeterminism by forcing it to encode individual letters) seemed like it helped: https://gwern.net/GPT-3-nonfiction#anagrams
For this task, I think a worthwhile followup would be to experiment with the new edit mode.
Possibly! Though it did seem to recognise that the words were spelt backwards. It must have some backwards spelt words in its training data, just not that many.