This strongly shows that GPT won’t be able to stay coherent with some initial state, which was already clear from it being autoregressive
This problem is not coming from the autoregressive part, if the dataset GPT was trained on contained a lot of examples of GPT making mistakes and then being corrected, it would be able to stay coherent for a long time (once it starts to make small deviations, it would immediately correct them because those small deviations were in the dataset, making it stable). This doesn’t apply to humans because humans don’t produce their actions by trying to copy some other agent, they learn their policy through interaction with the environment. So it’s not that a system in general is unable to stay coherent for long, but only those systems trained by pure imitation that aren’t able to do so.
Ok, now I understand better and I agree with this point, it’s like when you learn something faster if a teacher lets you try in small steps and corrects your errors at a granular level instead of leaving you alone in front of a large task you blankly stare at.
This problem is not coming from the autoregressive part, if the dataset GPT was trained on contained a lot of examples of GPT making mistakes and then being corrected, it would be able to stay coherent for a long time (once it starts to make small deviations, it would immediately correct them because those small deviations were in the dataset, making it stable). This doesn’t apply to humans because humans don’t produce their actions by trying to copy some other agent, they learn their policy through interaction with the environment. So it’s not that a system in general is unable to stay coherent for long, but only those systems trained by pure imitation that aren’t able to do so.
Ok, now I understand better and I agree with this point, it’s like when you learn something faster if a teacher lets you try in small steps and corrects your errors at a granular level instead of leaving you alone in front of a large task you blankly stare at.
For a response to this, see my comment above.