“Mike is large → large is Mike Bob is cute → cute is”
Also works w/ numbers (but I had trouble getting it to reverse 3 digits at a time): ”3 6 → 6 3 2 88 ->”
Ignoring a zero
“1 + 1 = 0 + 2 2 + 2 = 0 + 4 3 + 3 = 0 +”
Which also worked when replacing 0 w/ “pig”, but changing it to “df” made it predict ” 5″ as the answer, which I think it just wants to count up from the previous answer 4.
Parallel structure w/ Independent
For each of the following, the model predicts a ”.” at the end.
I eat spaghetti, yet she eats pizza I slept, for I was sleepy I can eat, or I can sleep I love my dog, and my dog loves me I can neither eat, nor can I sleep
Some n-grams overpower this effect. In the above “yet she eats ice” will be followed by ” cream”. “I was sleepy, so I slept” will be followed by ” in”.
Three Items in a List
“She ate the cookies, cake” will be followed by ”,” and then ” and”.
[Note: the language modelling game and the gpt-2 small search tool a were very useful]
Thanks for contributing these! I’m not sure I understand the one about ignoring a zero: is the idea that it can not only do normal addition, but also addition in the format with a zero?
Reversing text w/ 1 example:
“Mike is large → large is Mike
Bob is cute → cute is”
Also works w/ numbers (but I had trouble getting it to reverse 3 digits at a time):
”3 6 → 6 3
2 88 ->”
Ignoring a zero
“1 + 1 = 0 + 2
2 + 2 = 0 + 4
3 + 3 = 0 +”
Which also worked when replacing 0 w/ “pig”, but changing it to “df” made it predict ” 5″ as the answer, which I think it just wants to count up from the previous answer 4.
Parallel structure w/ Independent
For each of the following, the model predicts a ”.” at the end.
I eat spaghetti, yet she eats pizza
I slept, for I was sleepy
I can eat, or I can sleep
I love my dog, and my dog loves me
I can neither eat, nor can I sleep
Some n-grams overpower this effect. In the above “yet she eats ice” will be followed by ” cream”. “I was sleepy, so I slept” will be followed by ” in”.
Three Items in a List
“She ate the cookies, cake” will be followed by ”,” and then ” and”.
[Note: the language modelling game and the gpt-2 small search tool a were very useful]
Thanks for contributing these! I’m not sure I understand the one about ignoring a zero: is the idea that it can not only do normal addition, but also addition in the format with a zero?