I like the word “atom” to refer to units inside an SAE
But they’re not atomic! See eg the phenomena of feature splitting, and the fact that UMAP finds structure between semantically similar features
(In fairness, atoms are also not very atomic)
Extremely valid, you’ve convinced me that atom is probably a bad term for this
I think atom would be a pretty good choice as well but I think the actual choice of terminology is less important than the making the distinction. I used latent here because that’s what we used in our paper
I like the word “atom” to refer to units inside an SAE
But they’re not atomic! See eg the phenomena of feature splitting, and the fact that UMAP finds structure between semantically similar features
(In fairness, atoms are also not very atomic)
Extremely valid, you’ve convinced me that atom is probably a bad term for this
I think atom would be a pretty good choice as well but I think the actual choice of terminology is less important than the making the distinction. I used latent here because that’s what we used in our paper