asher comments on GPT-2′s positional embedding matrix is a helix

[asher]

Oh shoot, yea. I’m probably just looking at the rotary embeddings, then. Forgot about that, thanks