Chloe Li comments on Linear encoding of character-level information in GPT-J token embeddings

[Chloe Li]

We show that linear probes can retrieve character-level information from embeddings and we perform interventional experiments to show that this information is used by the model to carry out character-level tasks.

These two links need permission to be accessed.

[mwatkins]

Thanks for pointing this out! They should work now.