The most relevant paper I know of comes out of data privacy concerns. See Extracting Training Data from Large Language Models, which defines “k-eidetic memorization” as a string that can be elicited by some prompt and appears in at most k documents in the training set. They find several examples of k=1 memorization, though the strings appear repeatedly in the source documents. Unfortunately their methodology is targeted towards high-entropy strings and so is not universal.
I have a related question I’ve been trying to operationalize. How well do GPT-3′s memories “generalize”? In other words, given some fact in the training data, how far out of the source distribution can GPT-3 “gain information” from that fact?
E.g. training: “Ixlthubs live in the water.” Test: does this affect the predicted likelihood of “Ixlthubs live in the Pacific”? What about “Ixlthubs cannot survive on land”? I’d consider this another interesting measure of sample efficiency/generalization performance. I’m attempting to put together a proposal for the BigScience project (some set of synthetic facts to sprinkle throughout the data), but it’s my first try at something like this and slow going.
This is great, thanks! Then I wonder what people mean, exactly, when they say current methods are sample-inefficient. k=1 memorization seems to be about as good as humans, and this with tiny artificial neural nets! (Even GPT-3 is a thousand times smaller than a human brain).
Your question is super interesting as well. If you make progress on answering it, I’d love to hear!
The most relevant paper I know of comes out of data privacy concerns. See Extracting Training Data from Large Language Models, which defines “k-eidetic memorization” as a string that can be elicited by some prompt and appears in at most k documents in the training set. They find several examples of k=1 memorization, though the strings appear repeatedly in the source documents. Unfortunately their methodology is targeted towards high-entropy strings and so is not universal.
I have a related question I’ve been trying to operationalize. How well do GPT-3′s memories “generalize”? In other words, given some fact in the training data, how far out of the source distribution can GPT-3 “gain information” from that fact?
E.g. training: “Ixlthubs live in the water.” Test: does this affect the predicted likelihood of “Ixlthubs live in the Pacific”? What about “Ixlthubs cannot survive on land”? I’d consider this another interesting measure of sample efficiency/generalization performance. I’m attempting to put together a proposal for the BigScience project (some set of synthetic facts to sprinkle throughout the data), but it’s my first try at something like this and slow going.
This is great, thanks! Then I wonder what people mean, exactly, when they say current methods are sample-inefficient. k=1 memorization seems to be about as good as humans, and this with tiny artificial neural nets! (Even GPT-3 is a thousand times smaller than a human brain).
Your question is super interesting as well. If you make progress on answering it, I’d love to hear!