That’s because spaces are common in text for humans. The substring ′ It’ is common. Whereas, the string ′ SolidGoldMagikarp’ does not appear in the github repository vitaliya linked, but instead it is prefixed by a slash. I doubt any other backend source would have the leading space and this class of explanation seems poor to me.
Leading spaces are extremely common in GPT tokens. ′ It’, ′ That’, ′ an’ and ′ has’ are all tokens, for example.
That’s because spaces are common in text for humans. The substring ′ It’ is common. Whereas, the string ′ SolidGoldMagikarp’ does not appear in the github repository vitaliya linked, but instead it is prefixed by a slash. I doubt any other backend source would have the leading space and this class of explanation seems poor to me.
Oh, I see what you mean now.