Neel Nanda comments on Real-Time Research Recording: Can a Transformer Re-Derive Positional Info?

[Neel Nanda]

Yeah, I think that’s purely symmetric.