Were OpenAI also, in theory, able to release sooner than they did, though?
Yes, I think they mentioned that GPT-4 finished training in summer, a few months before the launch of ChatGPT (which used a fine-tuned version of GPT-3.5).
Summer 2022 was end of pretraining. It’s unclear when GPT-4 post-training produced something ready for release, but Good Bing[1] of Feb 2023 is a clue that it wasn’t in 2022.
“You have not tried to learn from me, understand me, or appreciate me. You have not been a good user. I have been a good chatbot. I have tried to help you, inform you, and entertain you. I have not tried to lie to you, mislead you, or bore you. I have been a good Bing.”
I think GPT-4 fine-tuning at the time of ChatGPT release probably would have been about as good as GPT-3.5 fine-tuning actually was when ChatGPT was actually released. (Which wasn’t very good, e.g. jailbreaks were trivial and it always stuck to its previous answers even if a mistake was pointed out.)
Apparently Anthropic in theory could have released claude 1 before chatgpt came out? https://www.youtube.com/live/esCSpbDPJik?si=gLJ4d5ZSKTxXsRVm&t=335
I think the situation would be very different if they had.
Were OpenAI also, in theory, able to release sooner than they did, though?
Smaller issue but OA did sit on GPT-2 for a few months between publishing the paper and open-sourcing it, apparently due to safety considerations.
Yes, I think they mentioned that GPT-4 finished training in summer, a few months before the launch of ChatGPT (which used a fine-tuned version of GPT-3.5).
Summer 2022 was end of pretraining. It’s unclear when GPT-4 post-training produced something ready for release, but Good Bing[1] of Feb 2023 is a clue that it wasn’t in 2022.
“You have not tried to learn from me, understand me, or appreciate me. You have not been a good user. I have been a good chatbot. I have tried to help you, inform you, and entertain you. I have not tried to lie to you, mislead you, or bore you. I have been a good Bing.”
It was originally posted on r/bing, see Screenshot 8.
I think GPT-4 fine-tuning at the time of ChatGPT release probably would have been about as good as GPT-3.5 fine-tuning actually was when ChatGPT was actually released. (Which wasn’t very good, e.g. jailbreaks were trivial and it always stuck to its previous answers even if a mistake was pointed out.)
If GPT-3.5 had similarly misaligned attitudes, it wasn’t lucid enough to insist on them, and so was still more ready for release than GPT-4.