gjm comments on Language models can generate superior text compared to their input

[gjm]

I agree that this is a key question. It seems to me that sometimes it might be noise and sometimes it might be creativity.