This is certainly intriguing! I’m tentatively skeptical this is the right perspective though for understanding what LMs are doing. An important difference is that in physics and dynamical systems, we often have pretty simple transition rules and want to understand how these generate complex patterns when run forward. For language models, the transition rule is itself extremely complicated. And I have this sense that the dynamics that arise aren’t that much more complicated in some sense. So arguably what we want to understand is the language model itself, not treat it as a black box and see what kinds of trajectories this black box induces. Though this does depend on where most of the “cognitive work” happens: right now it’s inside the forward pass of the language model, but if we rely more and more on chain-of-thought reasoning, hierarchies of lots of interacting LLMs, or anything like that, maybe this will change. In any case, this objection doesn’t rule out that stuff like the Lyapunov exponents could become nice tools, it’s just why I think we probably won’t get deep insights into AI cognition using an approach like this.
Trying to move towards semantic space instead of just token space seems like the right move not just because it’s what’s ultimately more meaningful to us, but also because transition dynamics should be simpler in semantic space in some sense. If you consider all possible dynamics on token space, the one a LM actually implements isn’t really special in any way (except that it has very non-uniform next-token probabilities). In contrast, it seems like the dynamics should be much simpler to specify in the “right” semantic space.
This is certainly intriguing! I’m tentatively skeptical this is the right perspective though for understanding what LMs are doing. An important difference is that in physics and dynamical systems, we often have pretty simple transition rules and want to understand how these generate complex patterns when run forward. For language models, the transition rule is itself extremely complicated. And I have this sense that the dynamics that arise aren’t that much more complicated in some sense. So arguably what we want to understand is the language model itself, not treat it as a black box and see what kinds of trajectories this black box induces. Though this does depend on where most of the “cognitive work” happens: right now it’s inside the forward pass of the language model, but if we rely more and more on chain-of-thought reasoning, hierarchies of lots of interacting LLMs, or anything like that, maybe this will change. In any case, this objection doesn’t rule out that stuff like the Lyapunov exponents could become nice tools, it’s just why I think we probably won’t get deep insights into AI cognition using an approach like this.
Trying to move towards semantic space instead of just token space seems like the right move not just because it’s what’s ultimately more meaningful to us, but also because transition dynamics should be simpler in semantic space in some sense. If you consider all possible dynamics on token space, the one a LM actually implements isn’t really special in any way (except that it has very non-uniform next-token probabilities). In contrast, it seems like the dynamics should be much simpler to specify in the “right” semantic space.