Nisan comments on Transformers Represent Belief State Geometry in their Residual Stream

[Nisan]

I suppose if you had more hidden states than observables, you could distinguish hidden-state prediction from next-token prediction by the dimension of the fractal.