I think you should be careful to not mix an analogy and an isomorphism. I agree that there is a pretty natural analogy with the cancer case, but it falls far short of an isomorphism at the moment. You don’t have an argument to say that the mechanism used by cancer cells are similar to those creating mesa-optimizers, that the process creating them is similar, etc
I’m not saying that such a lower level correspondence doesn’t exist. Just that saying “Look, the very general idea is similar” is not a strong enough argument for such a correspondence.
All analogies rely on isomorphisms. They simply refer to shared patterns. A good analogy captures many structural regularities that are shared between two different things while a bad one captures only a few.
The field of complex adaptive systems (CADs) is dedicated to the study of structural regularities between various systems operating under similar constraints. Ant colony optimization and simulated annealing can be used to solve an extremely wide range of problems because there are many structural regularities to CADs.
I worry that a myopic focus will result in a lot of time wasted on lines of inquiry that have parallels in a number of different fields. If we accept that the problem of inner alignment can be formalized, it would be very surprising to find that the problem is unique in the sense that it has no parallels in nature. Especially considering the obvious general analogy to the problem of cancer which may or may not provide insight to the alignment problem.
I think you should be careful to not mix an analogy and an isomorphism. I agree that there is a pretty natural analogy with the cancer case, but it falls far short of an isomorphism at the moment. You don’t have an argument to say that the mechanism used by cancer cells are similar to those creating mesa-optimizers, that the process creating them is similar, etc
I’m not saying that such a lower level correspondence doesn’t exist. Just that saying “Look, the very general idea is similar” is not a strong enough argument for such a correspondence.
All analogies rely on isomorphisms. They simply refer to shared patterns. A good analogy captures many structural regularities that are shared between two different things while a bad one captures only a few.
The field of complex adaptive systems (CADs) is dedicated to the study of structural regularities between various systems operating under similar constraints. Ant colony optimization and simulated annealing can be used to solve an extremely wide range of problems because there are many structural regularities to CADs.
I worry that a myopic focus will result in a lot of time wasted on lines of inquiry that have parallels in a number of different fields. If we accept that the problem of inner alignment can be formalized, it would be very surprising to find that the problem is unique in the sense that it has no parallels in nature. Especially considering the obvious general analogy to the problem of cancer which may or may not provide insight to the alignment problem.