It is not obvious to me from reading that transcript (and the attendant commentary) that GPT-3 was even checking to see whether or not the parentheses were balanced. Nor that it “knows” (or has in any way encoded the idea) that the sequence of parentheses between the quotes contains all the information needed to decide between balanced versus unbalanced, and thus every instance of the same parentheses sequence will have the same answer for whether or not it is balanced.
Reasons:
By my count, “John” got 18 out of 32 right which is not too far off from the average you would expect from random chance.
Arthur indicated that GPT-3 had at some point “generated inaccurate feedback from the teacher” which he edited out of the final transcript, so it was not only when taking the student’s perspective that there were errors.
GPT-3 does not seem to have a consistent mental model of John’s cognitive abilities and learning rate. At the end John gets a question wrong (even though John has already been told the answer for that specific sequence). But earlier, GPT-3 outputs that “By the end of the lesson, John has answered all of your questions correctly” and that John “learned all the rules about parentheses” and learned “all of elementary mathematics” in a week (or a day).
I suppose one way to test this (especially if OpenAI can provide the same random seed as was used here and make this reproducible) would be to have input prompts written from John’s perspective asking the teacher questions as if trying to understand the lesson. If GPT-3 is just “play-acting” based on the expected level of understanding of the character speaking, I would expect it to exhibit a higher level of accuracy/comprehension (on average, over many iterations) when writing from the perspective of the teacher rather than the student.
It is not obvious to me from reading that transcript (and the attendant commentary) that GPT-3 was even checking to see whether or not the parentheses were balanced. Nor that it “knows” (or has in any way encoded the idea) that the sequence of parentheses between the quotes contains all the information needed to decide between balanced versus unbalanced, and thus every instance of the same parentheses sequence will have the same answer for whether or not it is balanced.
Reasons:
By my count, “John” got 18 out of 32 right which is not too far off from the average you would expect from random chance.
Arthur indicated that GPT-3 had at some point “generated inaccurate feedback from the teacher” which he edited out of the final transcript, so it was not only when taking the student’s perspective that there were errors.
GPT-3 does not seem to have a consistent mental model of John’s cognitive abilities and learning rate. At the end John gets a question wrong (even though John has already been told the answer for that specific sequence). But earlier, GPT-3 outputs that “By the end of the lesson, John has answered all of your questions correctly” and that John “learned all the rules about parentheses” and learned “all of elementary mathematics” in a week (or a day).
I suppose one way to test this (especially if OpenAI can provide the same random seed as was used here and make this reproducible) would be to have input prompts written from John’s perspective asking the teacher questions as if trying to understand the lesson. If GPT-3 is just “play-acting” based on the expected level of understanding of the character speaking, I would expect it to exhibit a higher level of accuracy/comprehension (on average, over many iterations) when writing from the perspective of the teacher rather than the student.