A human examining the program can know which words were part of a prompt and which were just now generated by the machine, but I doubt the activation function examines the equations that are GPT’s own code, contemplates their significance and infers that the most recent letters were generated by it, or were part of the prompt
As a tangent, I do believe it’s possible to tell if an output is generated by GPT in principle. The model itself could potentially do that as well by noticing high-surprise words according to itself (ie low probability tokens in the prompt). I’m unsure if GPT-3 could be prompted to do that now though.
As a tangent, I do believe it’s possible to tell if an output is generated by GPT in principle. The model itself could potentially do that as well by noticing high-surprise words according to itself (ie low probability tokens in the prompt). I’m unsure if GPT-3 could be prompted to do that now though.