I think talking about distinct systems, one is emergent of another, with separate objectives/values is a really confusing ontology. Much better to present this idea in terms of attractors/local energy minima in dynamical systems and/or “bad equilibria” in games. There are no two systems, there are just different levels of system description at which information is transferred (see https://royalsocietypublishing.org/doi/full/10.1098/rsta.2021.0150): these levels could impose counteracting optimisation gradients, but nevertheless a system may be stuck in a globally suboptimal state.
I think talking about distinct systems, one is emergent of another, with separate objectives/values is a really confusing ontology. Much better to present this idea in terms of attractors/local energy minima in dynamical systems and/or “bad equilibria” in games. There are no two systems, there are just different levels of system description at which information is transferred (see https://royalsocietypublishing.org/doi/full/10.1098/rsta.2021.0150): these levels could impose counteracting optimisation gradients, but nevertheless a system may be stuck in a globally suboptimal state.