I’m surprised nobody has yet replied that the two examples are both products of significant optimizers with relevant optimization targets.
Yes, this seems pretty important and relevant.
That being said, I think that that definition suggests that natural selection and/or the earth’s crust are downstream from an optimiser of the number of Holiday Inns, or that my liver is downstream from an optimiser from my income, both of which aren’t right.
Probably it’s important to relate ‘natural subgoals’ to some ideal definition—which offers some hope, since ‘subgoal’ is really a computational notion, so maybe investigation along these lines would offer a more computational characterisation of optimisation.
[EDIT: I made this comment longer and more contentful]
Okay, so another necessary condition for being downstream from an optimizer is being causally downstream. I’m sure there are other conditions, but the claim still feels like an important addition to the conversation.
Yes, this seems pretty important and relevant.
That being said, I think that that definition suggests that natural selection and/or the earth’s crust are downstream from an optimiser of the number of Holiday Inns, or that my liver is downstream from an optimiser from my income, both of which aren’t right.
Probably it’s important to relate ‘natural subgoals’ to some ideal definition—which offers some hope, since ‘subgoal’ is really a computational notion, so maybe investigation along these lines would offer a more computational characterisation of optimisation.
[EDIT: I made this comment longer and more contentful]
Okay, so another necessary condition for being downstream from an optimizer is being causally downstream. I’m sure there are other conditions, but the claim still feels like an important addition to the conversation.