FWIW, my sense is that the projects people do are very often taken from Neel’s concrete open problems and are often meant for skilling up.
In traditional sciences, people often spend years learning to think better and use tools on problems that are “toy” and no serious researcher would work on, because this is how you improve and get to that point. I don’t think people should necessarily take years to get there, but it’s worth considering that upskilling is the primary motivation for much work.
The implication is that problems from Neel’s list are easier than mining interpretability observations that directly test predictions of some of the theories of DNNs or Transformers. I’m not sure if anyone tried to look directly at this question (also, these are not disjoint lists).
The implication is that problems from Neel’s list are easier than mining interpretability observations that directly test predictions of some of the theories of DNNs or Transformers. I’m not sure if anyone tried to look directly at this question (also, these are not disjoint lists).