Gwern, have you actually tried Bing Chat yet? If it is GPT-4, then it’s a big disappointment compared to how unexpectedly good ChatGPT was. It fails on simple logic and math questions, just like ChatGPT. I don’t find the ability to retrieve text from the web to be too impressive—it’s low-laying fruit that was long expected. It’s probably half-baked simply because Microsoft is in a hurry because they have limited time to gain market share before Google integrates Bard.
I have not. I assumed it’d be a very nerfed ChatGPT. By the time it became clear that it was interesting, they’d already started locking it down and the waitlist was so long I guessed that by the time I registered a MS account & got access, all of the initial interesting behavior would have been patched out and become unreplicable. (Like how most of the ChatGPT jailbreaks no longer work and in practice, it’s dumber than launch.) Likewise, the interesting thing about retrieval is the unexpected dynamics of users manipulating search results to program Sydney, which is more work than I’m willing to do and may also be patched heavily. (It would also be interesting for inner-monologue reasoning but the turn-limits alone make that hard to poke at now.)
Gwern, have you actually tried Bing Chat yet? If it is GPT-4, then it’s a big disappointment compared to how unexpectedly good ChatGPT was. It fails on simple logic and math questions, just like ChatGPT. I don’t find the ability to retrieve text from the web to be too impressive—it’s low-laying fruit that was long expected. It’s probably half-baked simply because Microsoft is in a hurry because they have limited time to gain market share before Google integrates Bard.
I have not. I assumed it’d be a very nerfed ChatGPT. By the time it became clear that it was interesting, they’d already started locking it down and the waitlist was so long I guessed that by the time I registered a MS account & got access, all of the initial interesting behavior would have been patched out and become unreplicable. (Like how most of the ChatGPT jailbreaks no longer work and in practice, it’s dumber than launch.) Likewise, the interesting thing about retrieval is the unexpected dynamics of users manipulating search results to program Sydney, which is more work than I’m willing to do and may also be patched heavily. (It would also be interesting for inner-monologue reasoning but the turn-limits alone make that hard to poke at now.)