That’s true for a very weak level of “remembering”. Given how much a transformer updates from a single fine tuning example, I think it’s basically impossible to generate something like episodic memory that you can later refer to.
It’s far more likely that the model just made that up—its entire job is to make up text, so it’s not at all surprising that it is doing that.
But, fair point, on some sense there’s memory there.
Given how much a transformer updates from a single fine tuning example, I think it’s basically impossible to generate something like episodic memory that you can later refer to.
Oh, not impossible. Don’t you remember how angry people were over exactly this happening with GPT-2/3, because it ‘violates privacy’? Large Transformers can memorize data which has been seen once: most recently, PaLM
Figure 18(b) shows the memorization rate as a function of the number of times a training example was exactly seen in the training data. We can see that examples seen exactly once in the training have a memorization rate of 0.75% for our largest model, while examples seen more than 500 times have a memorization rate of over 40%. Note that reason why there are any examples with such a high duplication rate is that our training is only de-duplicated on full documents, and here we evaluate memorization on 100 token spans...Larger models have a higher rate of memorization than smaller models...The chance that an example will be memorized strongly correlates with its uniqueness in the training. Examples that are only seen once are much less likely to be memorized than examples that are seen many times. This is consistent with previous work (Lee et al., 2021; Kandpal et al., 2022; Carlini et al., 2022)
0.75% is way higher than 0% and represents what must be millions of instances (don’t see how to break down their ‘2.4%’ of 540 billion tokens being memorized down into the % memorized seen-once but must be big). So, it is possible already, larger models would do more it more often, and seems reasonable to guess that memorization would be even higher for unique data included in a finetuning dataset rather than simply appearing somewhere in the pretraining.
That’s true for a very weak level of “remembering”. Given how much a transformer updates from a single fine tuning example, I think it’s basically impossible to generate something like episodic memory that you can later refer to.
It’s far more likely that the model just made that up—its entire job is to make up text, so it’s not at all surprising that it is doing that.
But, fair point, on some sense there’s memory there.
Oh, not impossible. Don’t you remember how angry people were over exactly this happening with GPT-2/3, because it ‘violates privacy’? Large Transformers can memorize data which has been seen once: most recently, PaLM
0.75% is way higher than 0% and represents what must be millions of instances (don’t see how to break down their ‘2.4%’ of 540 billion tokens being memorized down into the % memorized seen-once but must be big). So, it is possible already, larger models would do more it more often, and seems reasonable to guess that memorization would be even higher for unique data included in a finetuning dataset rather than simply appearing somewhere in the pretraining.
See also https://bair.berkeley.edu/blog/2020/12/20/lmmem/ https://arxiv.org/abs/2202.06539 https://arxiv.org/abs/2107.06499 https://arxiv.org/abs/2106.15110