Let’s model an autoregressive transformer as a Boolean circuit or, for simpler presentation, a n-ary circuit with m inputs and 1 output.
Model the entire system the following way: Given some particular m length starting input:
circuit calculates the output token (/integer) from the input
appends calculated output token to the end of the inputword
deletes first token of input
go to 1
It’s easy to see that, strictly speaking, this system is not very powerful computationally: we have finite number of possible tokens (n) and finite length context window (m), so we only have finite possible states (n*m), therefore our model is as powerful as a finite state machine (it’s pretty much equivalent in its behaviour to a regular grammar only containing A → aB rules)
However, real life computers also have finite memory yet we never let that bother us!
How should we manually design our circuit to enable us to solve the most types of problems with an appropriate selection of the initial input?
I think one very straightforward solution is to simply emulate a computer with random access memory the following way:
Select some fixed instruction set with k instructions and from our n tokens choose k to correspond to these k instructions.
Select another k tokens from the remaining to denote that the given instruction is under execution.
design the circuit such that if the emulated computer’s memory is M_t (m element vector, M_{ti} is the ith token) after the execution of the t-th instruction, then our circuit should compute the following tokens (including the starting input) : M_{00}, M_{01}, M_{02}, .. M_{0m}, M_{10}, M_{11}, .., M_{1m}, M_{20}, …
This can be done efficiently with relatively few cicuit nodes and relatively low depth, but I don’t want to write down the construction.
It’s interesting to see that actual large autoregressive transformers on human language seem to be fitting this model more and more closely:
With GPT-3 (possibly GPT-2), it was shown that after an instruction is given in the initial prompt, the transformer can execute that instruction in its continuation (eg. translate this french sentence to english, french: Je mange une pomme, english: ). This corresponds to having a fixed instruction set in the above model (where the instruction set is in common english instead of singular tokens)
With ChatGPT-3.5 and even more with newer models, it was shown that chain of thought prompting works well for solving more complex problems than asking for a solution immediately. I think the newest models often don’t even require an explicit instruction to break their reasoning down into steps, they often do so anyway. I expect this behaviour to be more and more common as newer models get smarter and also, encounter more and more transformer/human interactions in their training set. This corresponds to iteratively calculating M_1, M_2, … according to the given instructions. However, at this point, the instructions and subsequent “memory snapshots” are all in the transformer’s context window.
Might we expect this to change? Will future models be able to notice when the initial prompt or some still relevant previous data is about to exit the context window and autonomously re-generate them and subsequently pick up the calculation where they left off? I expect they will! What do you think?
Random Musing on Autoregressive Transformers resulting from Taelin’s A::B Challenge
Let’s model an autoregressive transformer as a Boolean circuit or, for simpler presentation, a n-ary circuit with m inputs and 1 output.
Model the entire system the following way: Given some particular m length starting input:
circuit calculates the output token (/integer) from the input
appends calculated output token to the end of the inputword
deletes first token of input
go to 1
It’s easy to see that, strictly speaking, this system is not very powerful computationally: we have finite number of possible tokens (n) and finite length context window (m), so we only have finite possible states (n*m), therefore our model is as powerful as a finite state machine (it’s pretty much equivalent in its behaviour to a regular grammar only containing A → aB rules)
However, real life computers also have finite memory yet we never let that bother us!
How should we manually design our circuit to enable us to solve the most types of problems with an appropriate selection of the initial input?
I think one very straightforward solution is to simply emulate a computer with random access memory the following way:
Select some fixed instruction set with k instructions and from our n tokens choose k to correspond to these k instructions.
Select another k tokens from the remaining to denote that the given instruction is under execution.
design the circuit such that if the emulated computer’s memory is M_t (m element vector, M_{ti} is the ith token) after the execution of the t-th instruction, then our circuit should compute the following tokens (including the starting input) : M_{00}, M_{01}, M_{02}, .. M_{0m}, M_{10}, M_{11}, .., M_{1m}, M_{20}, …
This can be done efficiently with relatively few cicuit nodes and relatively low depth, but I don’t want to write down the construction.
It’s interesting to see that actual large autoregressive transformers on human language seem to be fitting this model more and more closely:
With GPT-3 (possibly GPT-2), it was shown that after an instruction is given in the initial prompt, the transformer can execute that instruction in its continuation (eg. translate this french sentence to english, french: Je mange une pomme, english: ). This corresponds to having a fixed instruction set in the above model (where the instruction set is in common english instead of singular tokens)
With ChatGPT-3.5 and even more with newer models, it was shown that chain of thought prompting works well for solving more complex problems than asking for a solution immediately. I think the newest models often don’t even require an explicit instruction to break their reasoning down into steps, they often do so anyway. I expect this behaviour to be more and more common as newer models get smarter and also, encounter more and more transformer/human interactions in their training set. This corresponds to iteratively calculating M_1, M_2, … according to the given instructions. However, at this point, the instructions and subsequent “memory snapshots” are all in the transformer’s context window.
Might we expect this to change? Will future models be able to notice when the initial prompt or some still relevant previous data is about to exit the context window and autonomously re-generate them and subsequently pick up the calculation where they left off? I expect they will! What do you think?