Agreed with John, with the caveat that I expect search processes + simple objectives to only emerge from massively multi-task training. If you’re literally training an AI just on smiling, TurnTrout is right that “a spread of situationally-activated computations” is more likely since you’re not getting any value from the generality of search.
The Deep Double Descent paper is a good reference for why gradient descent training in the overparametrized regime favors low complexity models, though I don’t know of explicit evidence for the conjecture that “explicit search + simple objectives” is actually lower complexity (in model space) than “bundle of heuristics”. Seems intuitive if model complexity is something close to Kolmogorov complexity, but would love to see an empirical investigation!
Thinking about this more, I think gradient descent (at least in the modern regime) probably doesn’t select for inner search processes, because it’s not actually biased towards low Kolmogorov complexity. More in my standalone post, and here’s a John Maxwell comment making a similar point.
Agreed with John, with the caveat that I expect search processes + simple objectives to only emerge from massively multi-task training. If you’re literally training an AI just on smiling, TurnTrout is right that “a spread of situationally-activated computations” is more likely since you’re not getting any value from the generality of search.
The Deep Double Descent paper is a good reference for why gradient descent training in the overparametrized regime favors low complexity models, though I don’t know of explicit evidence for the conjecture that “explicit search + simple objectives” is actually lower complexity (in model space) than “bundle of heuristics”. Seems intuitive if model complexity is something close to Kolmogorov complexity, but would love to see an empirical investigation!
Thinking about this more, I think gradient descent (at least in the modern regime) probably doesn’t select for inner search processes, because it’s not actually biased towards low Kolmogorov complexity. More in my standalone post, and here’s a John Maxwell comment making a similar point.