This is admittedly pretty trivial but I am 90% sure that if you prompt GPT4 with “Q: What is today’s date?” it will not answer correctly. I think something like this would literally be the least impressive thing that GPT4 won’t be able to do.
Are you really 90% sure on that? For example, LaMDA apparently has live web query access (a direction OA was also exploring with WebGPT), and could easily recognize that as a factual query worth a web query, and if you search Google for “what is today’s date?” it will of course spit back “Monday, August 22, 2022″, which even the stupidest LMs could make good use of. So your prediction would appear to boil down to “OA won’t do an obviously useful thing they already half-did and a competitor did do a year ago”.
Well, if we’re counting things like that, this thread becomes much less interesting. They can offload math queries to a Mathematica-like software or chess playing to included Stockfish but we already know software can do this and we’re interested in novel capabilities of language or multi-modal models.
The difference is that LaMDA/WebGPT are learning autonomously to make general use of tools (or tool AIs) provided to them as agent AIs, which is much more useful than giant piles of human-hand-engineered heuristics like in toys like Alexa or Wolfram Alpha. In my example, no one would have programmed it to know it should query Google for the current date, it has learned to exploit Google’s various features on its own, which is no more illegitimate than learning to call date (or a human learning to look at a clock, for that matter), and will extend to any other tools provided it like a Python REPL in inner-monologue work.
Sure, it would be useful, especially if they’re gunning it to become a general chatbot assistant to take on Alexa or Google Home. But recognizing a factual query and offloading it to Google has been done by these assistants for years and it’s not something that anybody would find impressive anymore, even if the classifier was a part of a larger net.
This is admittedly pretty trivial but I am 90% sure that if you prompt GPT4 with “Q: What is today’s date?” it will not answer correctly. I think something like this would literally be the least impressive thing that GPT4 won’t be able to do.
Are you really 90% sure on that? For example, LaMDA apparently has live web query access (a direction OA was also exploring with WebGPT), and could easily recognize that as a factual query worth a web query, and if you search Google for “what is today’s date?” it will of course spit back “Monday, August 22, 2022″, which even the stupidest LMs could make good use of. So your prediction would appear to boil down to “OA won’t do an obviously useful thing they already half-did and a competitor did do a year ago”.
Yeah ok 80%. I also do concede this is a very trivial thing, not like some “gotcha look at what stupid LMs can’t do no AGI until 2400”.
Well, if we’re counting things like that, this thread becomes much less interesting. They can offload math queries to a Mathematica-like software or chess playing to included Stockfish but we already know software can do this and we’re interested in novel capabilities of language or multi-modal models.
The difference is that LaMDA/WebGPT are learning autonomously to make general use of tools (or tool AIs) provided to them as agent AIs, which is much more useful than giant piles of human-hand-engineered heuristics like in toys like Alexa or Wolfram Alpha. In my example, no one would have programmed it to know it should query Google for the current date, it has learned to exploit Google’s various features on its own, which is no more illegitimate than learning to call
date(or a human learning to look at a clock, for that matter), and will extend to any other tools provided it like a Python REPL in inner-monologue work.Sure, it would be useful, especially if they’re gunning it to become a general chatbot assistant to take on Alexa or Google Home. But recognizing a factual query and offloading it to Google has been done by these assistants for years and it’s not something that anybody would find impressive anymore, even if the classifier was a part of a larger net.
The ability for the AI to use tools like that is both impressive and useful, though.
AFAICT OpenAI now includes the current date in the prompt, so I think this is wrong
Yep! I was wrong and this is false!