Any time the embeddings /ā residual stream vectors is used for anything, they are projected onto the surface of a nā1 dimensional hypersphere. This changes the geometry.
Any time the embeddings /ā residual stream vectors is used for anything, they are projected onto the surface of a nā1 dimensional hypersphere. This changes the geometry.