Deep learning optimises over network parameter space directly.
Evolution optimises over the genome, and our genome is highly compressed wrt e.g. exact synaptic connections and cell makeup of our brains.
Optimising over a configuration space vs optimising over programs that produces configurations drawn from said space[1].
That seems like a very important difference, and meaningfully affects the selection pressures exerted on the models[2].
Furthermore, evolution does its optimisation via unbounded consequentialism in the real world.
As far as I’m aware, the fitness horizon for the genes is indefinite and evaluations are based on the actual/exact consequences.
Modern ML techniques seem disanalogous to evolution on multiple, important levels.
And I get the sense that forecasts of AI timelines based on such analogies are mostly illegitimate.
(I could also just be an idiot; I don’t understand evolution or ML well.)
I don’t actually know what all these means for timelines, takeoff or whether deep learning scales to AGI.
But it seems plausible to me that the effective optimisation applied by evolution in creating humans is grossly underestimated (wrong meta order of magnitude).
By selecting on the genome instead of on parameter space, evolution is selecting heavily for minimising description length and hence highly compressed versions of generally capable agent policies. This seems to exert stronger effective selection pressure for policies that contain optimisers than is exerted in deep learning.
This might be a contributing factor wrt sample efficiency of human brains vs deep learning.
Something to consider:
Deep learning optimises over network parameter space directly.
Evolution optimises over the genome, and our genome is highly compressed wrt e.g. exact synaptic connections and cell makeup of our brains.
Optimising over a configuration space vs optimising over programs that produces configurations drawn from said space[1].
That seems like a very important difference, and meaningfully affects the selection pressures exerted on the models[2].
Furthermore, evolution does its optimisation via unbounded consequentialism in the real world.
As far as I’m aware, the fitness horizon for the genes is indefinite and evaluations are based on the actual/exact consequences.
Modern ML techniques seem disanalogous to evolution on multiple, important levels.
And I get the sense that forecasts of AI timelines based on such analogies are mostly illegitimate.
(I could also just be an idiot; I don’t understand evolution or ML well.)
I don’t actually know what all these means for timelines, takeoff or whether deep learning scales to AGI.
But it seems plausible to me that the effective optimisation applied by evolution in creating humans is grossly underestimated (wrong meta order of magnitude).
@daniel_eth pointed out that evolution is not only meta optimising (via the genome) over model parameters but also model architectures.
By selecting on the genome instead of on parameter space, evolution is selecting heavily for minimising description length and hence highly compressed versions of generally capable agent policies. This seems to exert stronger effective selection pressure for policies that contain optimisers than is exerted in deep learning.
This might be a contributing factor wrt sample efficiency of human brains vs deep learning.