First, URLs you provided doesn’t support your assertion that you created tokens, and second:
Like since its possible to create the tokens, is it possible that some researcher in OpenAI has a very peculiar reason to enable this complexity create such logic and inject these mechanics.
Occams Razor.
I think it’s ideal to not predict intention by OpenAI when accident will do.
I would lean on the idea that GPT3 found these patterns and figured it would be interesting to embedd these themes into these tokens
I don’t think you did what I suggested in the comments above, based on the dataset you linked. It looks like you fine tuned on leilan and petertodd tokens. (From the two pieces of information you linked, I cannot determine if leilan and petertodd already existed in GPT.)
Unless you’re saying that the tokens didn’t previously exist—you’re not creating the tokens—and even then, the proposal I made was that you tokenize some new tokens, but not actually supply any training data that uses those tokens.
If you tokenized leilan and petertodd and then fine tuned on those tokens, that’s a different process than I proposed.
If you tokenize new tokens and then fine tune on them, I expect the GPT to behave according to the training data supplied on the tokens. That’s just standard, expected behavior.
Unless you’re saying that the tokens didn’t previously exist—you’re not creating the tokens—and even then, the proposal I made was that you tokenize some new tokens, but not actually supply any training data that uses those tokens.
Hmmm let me try and add two new tokens to try, based on your premise. I didn’t want to repeat on outlining my assertion in the recent comment as they were mentioned in my this one. I utilized jungian archetypes of the mother, ouroboros, shadow and hero as thematic concepts for GPT 3.5 to create the 510 stories.
… I utilized jungian archetypes of the mother, ouroboros, shadow and hero as thematic concepts for GPT 3.5 to create the 510 stories.
These are tokens that would already exist in the GPT. If you fine tune new writing to these concepts, then your fine tuning will influence the GPT responses when those tokens are used. That’s to be expected.
Hmmm let me try and add two new tokens to try, based on your premise.
If you want to review, ping me direct. Offer stands if you need to compare your plan against my proposal. (I didn’t think that was necessary, but, if you fine tuned tokens that already exist… maybe I wasn’t clear enough in my prior comments. I’m happy to chat via DM to compare your test plan against what I was proposing.)
These tokens already exist. It’s not really creating a token like ” petertodd”. Leilan is a name but ” Leilan” isn’t a name, and the token isn’t associated with the name.
If you fine tune on an existing token that has a meaning, then I maintain you’re not really creating glitch tokens.
First, URLs you provided doesn’t support your assertion that you created tokens, and second:
Occams Razor.
I think it’s ideal to not predict intention by OpenAI when accident will do.
I don’t think you did what I suggested in the comments above, based on the dataset you linked. It looks like you fine tuned on leilan and petertodd tokens. (From the two pieces of information you linked, I cannot determine if leilan and petertodd already existed in GPT.)
Unless you’re saying that the tokens didn’t previously exist—you’re not creating the tokens—and even then, the proposal I made was that you tokenize some new tokens, but not actually supply any training data that uses those tokens.
If you tokenized leilan and petertodd and then fine tuned on those tokens, that’s a different process than I proposed.
If you tokenize new tokens and then fine tune on them, I expect the GPT to behave according to the training data supplied on the tokens. That’s just standard, expected behavior.
Hmmm let me try and add two new tokens to try, based on your premise. I didn’t want to repeat on outlining my assertion in the recent comment as they were mentioned in my this one. I utilized jungian archetypes of the mother, ouroboros, shadow and hero as thematic concepts for GPT 3.5 to create the 510 stories.
These are tokens that would already exist in the GPT. If you fine tune new writing to these concepts, then your fine tuning will influence the GPT responses when those tokens are used. That’s to be expected.
If you want to review, ping me direct. Offer stands if you need to compare your plan against my proposal. (I didn’t think that was necessary, but, if you fine tuned tokens that already exist… maybe I wasn’t clear enough in my prior comments. I’m happy to chat via DM to compare your test plan against what I was proposing.)
“gustavo”: 50259,
“philippa”: 50260
I’ll add these tokens to the mix and cleaned/reformatted the dataset to use gustavo and philippa instead of petertodd and Leilan. Will share results.
These tokens already exist. It’s not really creating a token like ” petertodd”. Leilan is a name but ” Leilan” isn’t a name, and the token isn’t associated with the name.
If you fine tune on an existing token that has a meaning, then I maintain you’re not really creating glitch tokens.
These are new tokens I just thought of yesterday. I’m looking at the results I ran last night.
Edit: apparently the token reverted to the original token library, will go a different route and tweak the code to accomodate the new tokens.