I have argued that foundation models are best regarded as digital wilderness, uncharted territory that needs to be explored, mapped, and then settled. Just what I mean by “settling a plot of land in the digital wilderness” is not clear. This is, after all, but a metaphor for something that no one has ever done before. Ultimately, though, I believe settling will involve creating an index over some region of the language model and using symbolic means to operate with and over that index.

To that end I have been exploring the LLM underlying ChatGPT by giving ChatGPT tasks where its responses to prompts give me clues about the underlying structure of the LLM. I have been doing this since December 1, 2022 and have accumulated enough clues so that they are resonating with one another and giving me intuitions about how to begin explicitly mapping some territory in the LLM.

Most, but not all, of the posts listed below, are about working papers I have uploaded to Academia.edu. As such they contain the abstract, table of contents, introduction and, of course, a link to the working paper.