It’s interesting that the execution environment has excess privileges. But you shouldn’t cite GPT-4 as claiming things. You asked it to hallucinate metadata about the execution environment, and it did. GPT-4 itself is not a source for anything that isn’t in its context window, and I don’t think these things were described in the prmpt.
Mainly, because the claims in this post weren’t presented as claims about what was present in the system prompt, and the post presents information as “GPT-4 claims” in a way that’s consistent with getting fooled by hallucinations and not consistent with having done the sort of work required to figure out what the prompt actually was.
Yup, to be clear, I never actually directly accessed the code interpreter’s prompt, so GPT-4′s claims about constraints could be (and I expect at least a third of them to be) hallucinated
It’s interesting that the execution environment has excess privileges. But you shouldn’t cite GPT-4 as claiming things. You asked it to hallucinate metadata about the execution environment, and it did. GPT-4 itself is not a source for anything that isn’t in its context window, and I don’t think these things were described in the prmpt.
What makes you think they’re not in the prompt
Mainly, because the claims in this post weren’t presented as claims about what was present in the system prompt, and the post presents information as “GPT-4 claims” in a way that’s consistent with getting fooled by hallucinations and not consistent with having done the sort of work required to figure out what the prompt actually was.
Yup, to be clear, I never actually directly accessed the code interpreter’s prompt, so GPT-4′s claims about constraints could be (and I expect at least a third of them to be) hallucinated