This makes sense, but I’d argue that ML and interpretability has even more of both of these properties. Something that makes it harder is that some of the high-level goals of understanding transformers are inherently pretty complex, and also it’s less susceptible to math/ elegance-based analysis, so is even more messy :)
This makes sense, but I’d argue that ML and interpretability has even more of both of these properties. Something that makes it harder is that some of the high-level goals of understanding transformers are inherently pretty complex, and also it’s less susceptible to math/ elegance-based analysis, so is even more messy :)