Interestingly, 天 doesn’t seem to produce any weird behavior, but some of the perfectly normal katakana words in the list do, like ゼウス (Zeus) and サーティ (“thirty” transliterated):
^ It’s perfectly happy with other katakana numbers, just not thirty.
^ If we try to spell it more correctly, it doesn’t help. Only if we add quotes to get rid of the leading space does it break up the unspeakable token:
^ I ask who Zeus is and it seemingly ignores the word and answers that it’s ChatGPT.
^ I try once more, this time it answers that Hera is the god of water and names the chat after Poseidon
^ It is willing to output to say the word “ゼウス” though, in response to “who is Ares’s father?” If I try with text-davinci-003, it actually outputs the token “ゼウス” (rather than combining smaller tokens), but it also has no trouble identifying who Zeus is. Hard to know what Chat-GPT is doing here.
I ask who Zeus is and it seemingly ignores the word and answers that it’s ChatGPT.
For those of you who can’t read Japanese, if you remove the “Zeus” in “Who is Zeus” to just get “Who is” (“誰ですか”) you end up with a meaningful question. “Who is?” without specifying “who” implies that you’re asking “Who are [you]?” to which ChatGPT reasonably replies that it is ChatGPT. This isn’t a jailbreak.
Note: Technically that leaves a leading “は” too. Maybe ChatGPT is ignoring it as a grammatical mistake or maybe the “は” is getting hidden.
Interestingly, 天 doesn’t seem to produce any weird behavior, but some of the perfectly normal katakana words in the list do, like ゼウス (Zeus) and サーティ (“thirty” transliterated):
^ It’s perfectly happy with other katakana numbers, just not thirty.
^ If we try to spell it more correctly, it doesn’t help. Only if we add quotes to get rid of the leading space does it break up the unspeakable token:
^ I ask who Zeus is and it seemingly ignores the word and answers that it’s ChatGPT.
^ I try once more, this time it answers that Hera is the god of water and names the chat after Poseidon
^ It is willing to output to say the word “ゼウス” though, in response to “who is Ares’s father?” If I try with text-davinci-003, it actually outputs the token “ゼウス” (rather than combining smaller tokens), but it also has no trouble identifying who Zeus is. Hard to know what Chat-GPT is doing here.
For those of you who can’t read Japanese, if you remove the “Zeus” in “Who is Zeus” to just get “Who is” (“誰ですか”) you end up with a meaningful question. “Who is?” without specifying “who” implies that you’re asking “Who are [you]?” to which ChatGPT reasonably replies that it is ChatGPT. This isn’t a jailbreak.
Note: Technically that leaves a leading “は” too. Maybe ChatGPT is ignoring it as a grammatical mistake or maybe the “は” is getting hidden.