since it’s near-impossible to identify which specific heuristic the model is using (there can always be a slightly more complex, more general heuristic of which your chosen heuristic is a special case).
I’m putting some of my faith in low-rank decompositions of bilinear MLPs but I’ll let you know if I make any real progress with it :)
I’m putting some of my faith in low-rank decompositions of bilinear MLPs but I’ll let you know if I make any real progress with it :)