This is the jacobian taken at a single data point, right? Assuming so, you might want to try looking for a single rotation which makes the jacobian sparse at many datapoints simultaneously. That would be more directly interpretable as factoring the net into a relatively low number of information channels.
Another useful next step would be to take some part of the net which maps X → Y → Z, and compute the rotations which maximize sparsity for X → Y and Y → Z separately. Then, try to compose the rotations found. Do the “sparse output channels” of X → Y approximately match the “sparse input channels” of Y → Z?
I was planning on doing the first idea, and I do like the second idea! I’m slightly skeptical that the two rotations will be the same, but I did find that when performing the method on the last layer of the model, I get the identity matrix, which is some evidence in favor of the ‘rotations are the same’ prediction being right in general.
I think the idea is that if the rotated basis fundamentally “means” something important, rather than just making what’s happening easier to picture for us humans, we’d kind of expect the basis computed for X->Y to mostly match the basis for Y->Z.
At least that’s the sort of thing I’d expect to see in such a world.
Yup, this is why I’m skeptical there will be a positive result. I did not try to derive a principled, meaningful, basis. I tried the most obvious thing to do which nobody else seems to have done. So I expect this device will be useful and potentially the start of something fundamental, but not fundamental itself.
This is the jacobian taken at a single data point, right? Assuming so, you might want to try looking for a single rotation which makes the jacobian sparse at many datapoints simultaneously. That would be more directly interpretable as factoring the net into a relatively low number of information channels.
Another useful next step would be to take some part of the net which maps X → Y → Z, and compute the rotations which maximize sparsity for X → Y and Y → Z separately. Then, try to compose the rotations found. Do the “sparse output channels” of X → Y approximately match the “sparse input channels” of Y → Z?
I was planning on doing the first idea, and I do like the second idea! I’m slightly skeptical that the two rotations will be the same, but I did find that when performing the method on the last layer of the model, I get the identity matrix, which is some evidence in favor of the ‘rotations are the same’ prediction being right in general.
I think the idea is that if the rotated basis fundamentally “means” something important, rather than just making what’s happening easier to picture for us humans, we’d kind of expect the basis computed for X->Y to mostly match the basis for Y->Z.
At least that’s the sort of thing I’d expect to see in such a world.
Yup, this is why I’m skeptical there will be a positive result. I did not try to derive a principled, meaningful, basis. I tried the most obvious thing to do which nobody else seems to have done. So I expect this device will be useful and potentially the start of something fundamental, but not fundamental itself.