Agree. I like to split the empirical problems out using levels of abstraction:
Traversal problems: each experiment is expensive or it isn’t clear how to generate a new experiment from old ones because of lack of visibility about controlled variables.
Representation space problems: the search space is too large, our experiments don’t reliably screen off large portions of it. So we can’t expect to converge in any reasonable time.
Intentional problems: we’re not even clear on what we’re trying to do or whether our representation of what we’re trying to do matches the natural categories of the solution space such that we are even testing real things when we design the experiment.
Implementation problems: we can’t build the tooling or control the variable we need to control even if we are pretty sure what it is. Measurement problems means we can’t distinguish between important final or intermediate outcomes (eg error bars).
It is often characterized as 3 levels but if you read his book the algorithmic level is split into traversal and representation (which is highly useful as a way to think about algorithms in general) As four levels it also corresponds to Aristotle’s 4 whys: final, formal, efficient, and material.
So intentional problems would be markets, where noise is being injected and any clear pattern is being drained dry by automated systems, preventing you from converging to a model. Or public private encryption where you aren’t supposed to be able to solve it? (But possibly you can)
Agree. I like to split the empirical problems out using levels of abstraction:
Traversal problems: each experiment is expensive or it isn’t clear how to generate a new experiment from old ones because of lack of visibility about controlled variables.
Representation space problems: the search space is too large, our experiments don’t reliably screen off large portions of it. So we can’t expect to converge in any reasonable time.
Intentional problems: we’re not even clear on what we’re trying to do or whether our representation of what we’re trying to do matches the natural categories of the solution space such that we are even testing real things when we design the experiment.
Implementation problems: we can’t build the tooling or control the variable we need to control even if we are pretty sure what it is. Measurement problems means we can’t distinguish between important final or intermediate outcomes (eg error bars).
Does the phrase “levels of abstraction” imply that those four problems form some kind of hierarchy? If so, could you explain how that hierarchy works?
https://en.wikipedia.org/wiki/David_Marr_(neuroscientist)#Levels_of_analysis
It is often characterized as 3 levels but if you read his book the algorithmic level is split into traversal and representation (which is highly useful as a way to think about algorithms in general) As four levels it also corresponds to Aristotle’s 4 whys: final, formal, efficient, and material.
So intentional problems would be markets, where noise is being injected and any clear pattern is being drained dry by automated systems, preventing you from converging to a model. Or public private encryption where you aren’t supposed to be able to solve it? (But possibly you can)