Thank you Abram. Yes, factored cognition is more what I had in mind. However, I think it’s possible to speak of decomposition generally enough to say that PCA/SVD is a decomposer, albeit an incredibly parochial one that’s not very useful to factored cognition.
Like, my read of IDA is that the distillation step is proposing a class of algorithms, and we may find that SVD was a member of that class all along.
I was asking why because I wanted to understand what you mean by “decomposition”.
Defines many things.
Usually the goal is feature extraction (think Bert) or reducing the size of a representation (think autoencoders or simpler , PCA)
You need to narrow down your definition, I think, to get a meaningful answers.
I think Quinn means factored cognition, which is quite different from autoencoders/embeddings/PCA.
Thank you Abram. Yes, factored cognition is more what I had in mind. However, I think it’s possible to speak of decomposition generally enough to say that PCA/SVD is a decomposer, albeit an incredibly parochial one that’s not very useful to factored cognition.
Like, my read of IDA is that the distillation step is proposing a class of algorithms, and we may find that SVD was a member of that class all along.