Does OpenAI say this, or are you inferring it entirely from the Wolfram blog post? Isn’t that an odd place to learn such a thing?
And where does the Wolfram blog post say this? It sounds to me like he’s doing something like this outsider, making one call to Wolfram, then using the LLM to evaluate the result and determine if it produced an error and retry.
I am inferring this because plugins would simply not work otherwise.
Please think about what it would mean for each “plugin supported query” for the AI to have to read all of the tokens of all of the plugins. Remember every piece of information OAI doesn’t put into the model weights costs you tokens from your finite length context window. Remember you can go look at the actual descriptions of many plugins and they eat 1000+ tokens alone, or 1⁄8 your window to remember what one plugin does.
Or that what it would cost OAI to keep generating GPT-4 tokens again and again and again for the machine to fail to make a request over and over and over. Or for a particular plugin to essentially lie in it’s description and be useless. Or the finer points of when to search bing vs wolfram alpha, for example for pokemon evolutions and math, wolfram, but for current news, bing...
I’m pretty confident that I have been using the “Plugins” model with a very long context window. I was copy-pasting entire 500-line source files and asking questions about it. I assume that I’m getting the 32k context window.
The entire conversation is over 60,000 characters according to wc. OpenAI’s tool won’t even let me compute the tokens if I paste more than 50k (?) characters, but when I deleted some of it, it gave me a value of >18,000 tokens.
I’m not sure if/when ChatGPT starts to forgot part of the chat history (drops out of the context window) but it still seemed to remember the first file after long, winding discussion.
Since you have to manually activate plugins, they don’t take any context until you do so. In particular, multiple plugins don’t compete for context and the machine doesn’t decide which one to use.
Please read the documentation and the blog post you cited.
Does OpenAI say this, or are you inferring it entirely from the Wolfram blog post? Isn’t that an odd place to learn such a thing?
And where does the Wolfram blog post say this? It sounds to me like he’s doing something like this outsider, making one call to Wolfram, then using the LLM to evaluate the result and determine if it produced an error and retry.
I am inferring this because plugins would simply not work otherwise.
Please think about what it would mean for each “plugin supported query” for the AI to have to read all of the tokens of all of the plugins. Remember every piece of information OAI doesn’t put into the model weights costs you tokens from your finite length context window. Remember you can go look at the actual descriptions of many plugins and they eat 1000+ tokens alone, or 1⁄8 your window to remember what one plugin does.
Or that what it would cost OAI to keep generating GPT-4 tokens again and again and again for the machine to fail to make a request over and over and over. Or for a particular plugin to essentially lie in it’s description and be useless. Or the finer points of when to search bing vs wolfram alpha, for example for pokemon evolutions and math, wolfram, but for current news, bing...
I’m pretty confident that I have been using the “Plugins” model with a very long context window. I was copy-pasting entire 500-line source files and asking questions about it. I assume that I’m getting the 32k context window.
How many characters is your 500 line source file? It probably fits in 8k tokens. You can find out here
The entire conversation is over 60,000 characters according to wc. OpenAI’s tool won’t even let me compute the tokens if I paste more than 50k (?) characters, but when I deleted some of it, it gave me a value of >18,000 tokens.
I’m not sure if/when ChatGPT starts to forgot part of the chat history (drops out of the context window) but it still seemed to remember the first file after long, winding discussion.
Since you have to manually activate plugins, they don’t take any context until you do so. In particular, multiple plugins don’t compete for context and the machine doesn’t decide which one to use.
Please read the documentation and the blog post you cited.
“An experimental model that knows when and how to use plugins”
Sounds like they updated the model.
And it says you have to activate third party plugins. Browser, python interpreter will probably always be active.
That’s rather useless then.