I don’t know to what extent researchers themselves agree with this point—it seems like there is a lot of adversarial examples research that is looking at the imperceptible perturbation case and many papers that talk about new types of adversarial examples, without really explaining why they are doing this or giving a motivation that is about unsolved research problems rather than real world settings. It’s possible that researchers do think of it as a research problem and not a real world problem, but present their papers differently because they think that’s necessary in order to be accepted.
The distinction between research problems and real world threat models seem to parallel the distinction between theoretical or conceptual research and engineering in AI safety. (I would include not just Agent Foundations in the former, but also a lot of CHAI’s work, e.g. CIRL, so I don’t know if this is the same thing you’re pointing at.) The former typically asks questions of the form “how could we do this in principle, making simplifying assumptions X, Y and Z”, even though X, Y and Z are known not to hold in the real world, for the sake of having greater conceptual clarity that can later be leveraged as a solution to a real world problem. Engineering work on the other hand is typically trying to scale an approach to a more complex environment (with the eventual goal of getting to a real world problem).
I don’t know to what extent researchers themselves agree with this point—it seems like there is a lot of adversarial examples research that is looking at the imperceptible perturbation case and many papers that talk about new types of adversarial examples, without really explaining why they are doing this or giving a motivation that is about unsolved research problems rather than real world settings. It’s possible that researchers do think of it as a research problem and not a real world problem, but present their papers differently because they think that’s necessary in order to be accepted.
The distinction between research problems and real world threat models seem to parallel the distinction between theoretical or conceptual research and engineering in AI safety. (I would include not just Agent Foundations in the former, but also a lot of CHAI’s work, e.g. CIRL, so I don’t know if this is the same thing you’re pointing at.) The former typically asks questions of the form “how could we do this in principle, making simplifying assumptions X, Y and Z”, even though X, Y and Z are known not to hold in the real world, for the sake of having greater conceptual clarity that can later be leveraged as a solution to a real world problem. Engineering work on the other hand is typically trying to scale an approach to a more complex environment (with the eventual goal of getting to a real world problem).