Hm, I mostly agree. There isn’t any interesting structure by default, you have to get it by trying to mimic a training distribution that has interesting structure.
And I think this relates to another way that I was too reductive, which is that if I want to talk about “simulacra” as a thing, then they don’t exist purely in the text, so I must be sneaking in another ontology somewhere—an ontology that consists of features inferred from text (but still not actually the state of our real universe).
Hm, I mostly agree. There isn’t any interesting structure by default, you have to get it by trying to mimic a training distribution that has interesting structure.
And I think this relates to another way that I was too reductive, which is that if I want to talk about “simulacra” as a thing, then they don’t exist purely in the text, so I must be sneaking in another ontology somewhere—an ontology that consists of features inferred from text (but still not actually the state of our real universe).