…So that’s all that’s needed. If any system has both a capacity for endogenous action (motor control, attention control, etc.), and a generic predictive learning algorithm, that algorithm will be automatically incentivized to develop generative models about itself (both its physical self and its algorithmic self), in addition to (and connected to) models about the outside world.
Yes, and there are many different classes of such models. Most of them boring because the prediction of the effect of the agent on the environment is limited (small effect or low data rate) or simple (linear-ish or more-is-better-like).
But the self-models of social animals will quickly grow complex because the prediction of the action on the environment includes elements in the environment—other members of the species—that themselves predict the actions of other members.
You don’t mention it, but I think Theory of Mind or Emphatic Inference play a large role in the specific flavor of human self-models.
Yes, and there are many different classes of such models. Most of them boring because the prediction of the effect of the agent on the environment is limited (small effect or low data rate) or simple (linear-ish or more-is-better-like).
But the self-models of social animals will quickly grow complex because the prediction of the action on the environment includes elements in the environment—other members of the species—that themselves predict the actions of other members.
You don’t mention it, but I think Theory of Mind or Emphatic Inference play a large role in the specific flavor of human self-models.