Thanks, I found the post quite stimulating. Some questions and thoughts:
Is LLM dynamics ergodic? I.e. is the time average P∞ equal to limN→∞1N∑Nnπn(0), the average page vector?.
One potential issue with this formalisation is that you always assume a prompt of size k (so you need to introduce artificial “null tokens” if the prompt is shorter) and you don’t give special treatment to the token <|endoftext|>. For me, it would be more intuitive to consider LLM dynamics in terms of finite, variable length, token-level Markov chains (until <|endoftext|>). While a fixed block size is actually being used during training, the LLM is incentivised to disregard anything before <|endoftext|>. So these two prompts should induce the same distribution:
Document about cats.<|endoftext|>My name is;
Document about dogs.<|endoftext|>My name is.
Your formalisation doesn’t account for this symmetry.
Dennett is spelled with “tt”.
Note that a softmax-based LLM will always put non-zero probability on every token. So there are no strictly absorbing states. You’re careful enough to define absorbing states as “once you enter, you are unlikely to ever leave”, but then your toy Waluigi model is implausible. A Waluigi can always switch back to a Luigi.
Thanks, I found the post quite stimulating. Some questions and thoughts:
Is LLM dynamics ergodic? I.e. is the time average P∞ equal to limN→∞1N∑Nnπn(0), the average page vector?.
One potential issue with this formalisation is that you always assume a prompt of size k (so you need to introduce artificial “null tokens” if the prompt is shorter) and you don’t give special treatment to the token
<|endoftext|>. For me, it would be more intuitive to consider LLM dynamics in terms of finite, variable length, token-level Markov chains (until<|endoftext|>). While a fixed block size is actually being used during training, the LLM is incentivised to disregard anything before<|endoftext|>. So these two prompts should induce the same distribution:Document about cats.<|endoftext|>My name is;Document about dogs.<|endoftext|>My name is. Your formalisation doesn’t account for this symmetry.Dennett is spelled with “tt”.
Note that a softmax-based LLM will always put non-zero probability on every token. So there are no strictly absorbing states. You’re careful enough to define absorbing states as “once you enter, you are unlikely to ever leave”, but then your toy Waluigi model is implausible. A Waluigi can always switch back to a Luigi.
Almost certainly ergodic in the limit. But it’s highly period due to English grammar.
Yep, just for convenience.
Yep.
Temp = 0 would give exactly absorbing states.