If however people use + to mean addition GPT3 is already capable enough to learn the concept and use it to add numbers that aren’t in it’s training corpus.
Yes, but it will still be about as good as its training corpus.
One way of looking at this is that GPT-X is trying to produce text that looks just like human written text. Given two passages of text, there should be no easy way to tell which was written by a human, and which wasn’t.
GPT-X has expertise in all subjects, in a sense. Each time it produces text, it is sampling from the distribution of human competence. Detailed information about anteaters is in there somewhere, every now and again, it will sample an expert on them, but most of the time it will act like a person who doesn’t know much about anteaters.
Yes, but it will still be about as good as its training corpus.
One way of looking at this is that GPT-X is trying to produce text that looks just like human written text. Given two passages of text, there should be no easy way to tell which was written by a human, and which wasn’t.
GPT-X has expertise in all subjects, in a sense. Each time it produces text, it is sampling from the distribution of human competence. Detailed information about anteaters is in there somewhere, every now and again, it will sample an expert on them, but most of the time it will act like a person who doesn’t know much about anteaters.