Isn’t evolution a better analogy for deep learning anyway? All natural selection does is gradient descent (hill climbing technically), with no capacity for lookahead. And we’ve known this one for 150 years!
All natural selection does is gradient descent (hill climbing technically), with no capacity for lookahead.
I think if you’re interested in the analysis and classification of optimization techniques, there’s enough differences between what natural selection is doing and what deep learning is doing that it isn’t a very natural analogy. (Like, one is a population-based method and the other isn’t, the update rules are different, etc.)
Isn’t evolution a better analogy for deep learning anyway? All natural selection does is gradient descent (hill climbing technically), with no capacity for lookahead. And we’ve known this one for 150 years!
I think if you’re interested in the analysis and classification of optimization techniques, there’s enough differences between what natural selection is doing and what deep learning is doing that it isn’t a very natural analogy. (Like, one is a population-based method and the other isn’t, the update rules are different, etc.)