This leads to a natural question: What reflection process would change a language model towards becoming a better map of the world (rather than language in the training dataset)? Reflection only looks at the language model, doesn’t look at the world, produces an improved version of the model, applies an inductive bias after the fact. This is a problem statement of epistemic rationality for AI.
That’s not reflection, just more initial training data. Reflection acts on the training data it already has, the point is to change the learning problem, by introducing an inductive bias that’s not part of the low level learning algorithm, that improves sample efficiency with respect to loss that’s also not part of low level learning. LLMs are a very good solution to the wrong problem, and a so-so solution to the right problem. Changing the learning incentives might get a better use out of the same training data for improving performance on the right problem.
A language model retrained on generated text (which is one obvious form of implementing reflection) likely does worse as a language model of the original training data, it’s only a better model of the original data with respect to some different metric of being a good model (such as being a good map of the actual world, whatever that means). Machine learning doesn’t know how to specify or turn this different metric into a learning algorithm, but an amplification process that makes use of faculties an LLM captured from human use of language might manage to do this by generating appropriate text for low level learning.
I think in the limit of text prediction, language models can learn ~all of humanity’s shared world model that is represented explicitly. The things that language models can’t learn are IMO:
Tacit knowledge of the world that we haven’t represented in text
Underdetermined features of the world
Aspects of our shared world model as represented in language that do not uniquely constrain our particular universe
This leads to a natural question: What reflection process would change a language model towards becoming a better map of the world (rather than language in the training dataset)? Reflection only looks at the language model, doesn’t look at the world, produces an improved version of the model, applies an inductive bias after the fact. This is a problem statement of epistemic rationality for AI.
At a guess, focusing on transforming information from images and videos into text, rather than generating text qua text, ought to help — no?
That’s not reflection, just more initial training data. Reflection acts on the training data it already has, the point is to change the learning problem, by introducing an inductive bias that’s not part of the low level learning algorithm, that improves sample efficiency with respect to loss that’s also not part of low level learning. LLMs are a very good solution to the wrong problem, and a so-so solution to the right problem. Changing the learning incentives might get a better use out of the same training data for improving performance on the right problem.
A language model retrained on generated text (which is one obvious form of implementing reflection) likely does worse as a language model of the original training data, it’s only a better model of the original data with respect to some different metric of being a good model (such as being a good map of the actual world, whatever that means). Machine learning doesn’t know how to specify or turn this different metric into a learning algorithm, but an amplification process that makes use of faculties an LLM captured from human use of language might manage to do this by generating appropriate text for low level learning.
We could do auto captioning of movies and videos.
Or we could just train multimodal simulators. We probably will (e.g. such models could be useful for generating videos from descriptions).
I think in the limit of text prediction, language models can learn ~all of humanity’s shared world model that is represented explicitly. The things that language models can’t learn are IMO:
Tacit knowledge of the world that we haven’t represented in text
Underdetermined features of the world
Aspects of our shared world model as represented in language that do not uniquely constrain our particular universe
Stuff we don’t know about the world