Oh cool. LMs can output more finely tokenized text than it’s trained on, so it probably didn’t output the token ” Skydragon”, but instead multiple tokens, [” ”, “Sky”, “dragon”] or something
Oh cool. LMs can output more finely tokenized text than it’s trained on, so it probably didn’t output the token ” Skydragon”, but instead multiple tokens, [” ”, “Sky”, “dragon”] or something