I think a bacterium is not an optimizer. Rather, it is optimized by evolution. Animals start being optimizers by virtue of planning over internal representations of external states, which makes them mesaoptimizers of evolution.
If we follow this model, we may consider that optimization requires a map-territory distinction. in that view, DNA is the map of evolution, and the CNS is the map of the animal. If the analogy holds, I’d speculate that the weights are the map of reinforcement learning, and the context window is the map of the mesaoptimizer.
I think a bacterium is not an optimizer. Rather, it is optimized by evolution. Animals start being optimizers by virtue of planning over internal representations of external states, which makes them mesaoptimizers of evolution.
Hmm, so where does the “true” optimization start? Or, at least what is the range of living creatures which are not-quite-complex to count as optimizers? Clearly a fish would be one, right? What about a sea cucumber? A plant?
Hm, difficult. I think the minimal required trait is the ability to learn patterns that map outputs to deferred reward inputs. So an organism that simply reacts to inputs directly would not be an optimizer, even if it has a (static) nervous system. A test may be if the organism can be made to persistedly change strategy by a change in reward, even in the immediate absence of the reward signal.
I think maybe you could say that ants are not anthill optimizers? Because the optimization mechanism doesn’t operate at all on the scale of individual ants? Not sure if that holds up.
I think a bacterium is not an optimizer. Rather, it is optimized by evolution. Animals start being optimizers by virtue of planning over internal representations of external states, which makes them mesaoptimizers of evolution.
If we follow this model, we may consider that optimization requires a map-territory distinction. in that view, DNA is the map of evolution, and the CNS is the map of the animal. If the analogy holds, I’d speculate that the weights are the map of reinforcement learning, and the context window is the map of the mesaoptimizer.
Hmm, so where does the “true” optimization start? Or, at least what is the range of living creatures which are not-quite-complex to count as optimizers? Clearly a fish would be one, right? What about a sea cucumber? A plant?
Hm, difficult. I think the minimal required trait is the ability to learn patterns that map outputs to deferred reward inputs. So an organism that simply reacts to inputs directly would not be an optimizer, even if it has a (static) nervous system. A test may be if the organism can be made to persistedly change strategy by a change in reward, even in the immediate absence of the reward signal.
I think maybe you could say that ants are not anthill optimizers? Because the optimization mechanism doesn’t operate at all on the scale of individual ants? Not sure if that holds up.