MiguelDev comments on Mech Interp Puzzle 1: Suspiciously Similar Embeddings in GPT-Neo

MiguelDev 17 Jul 2023 0:52 UTC
1 point
0
@Neel Nanda but isn’t it that there are more layers that produce the actual vocab tokens? it’s not only layer 0 or the embedding layer right?