Marius Hobbhahn comments on LLM Basics: Embedding Spaces—Transformer Token Vectors Are Not Points in Space

[Marius Hobbhahn]

fair. You convinced me that the effect is more determined by layer-norm than cross-entropy.