It’s a fascinating phenomenon. If I had to bet I would say it isn’t a coping mechanism but rather a particular manifestation of a deeper inductive bias of the learning process.
That’s a really interesting blogpost, thanks for sharing! I skimmed it but I didn’t really grasp the point you were making here. Can you explain what you think specifically causes self-repair?
I think self-repair might have lower free energy, in the sense that if you had two configurations of the weights, which “compute the same thing” but one of them has self-repair for a given behaviour and one doesn’t, then the one with self-repair will have lower free energy (which is just a way of saying that if you integrate the Bayesian posterior in a neighbourhood of both, the one with self-repair gives you a higher number, i.e. its preferred).
That intuition is based on some understanding of what controls the asymptotic (in the dataset size) behaviour of the free energy (which is -log(integral of posterior over region)) and the example in that post. But to be clear it’s just intuition. It should be possible to empirically check this somehow but it hasn’t been done.
Basically the argument is self-repair ⇒ robustness of behaviour to small variations in the weights ⇒ low local learning coefficient ⇒ low free energy ⇒ preferred
I think by “specifically” you might be asking for a mechanism which causes the self-repair to develop? I have no idea.
It’s a fascinating phenomenon. If I had to bet I would say it isn’t a coping mechanism but rather a particular manifestation of a deeper inductive bias of the learning process.
That’s a really interesting blogpost, thanks for sharing! I skimmed it but I didn’t really grasp the point you were making here. Can you explain what you think specifically causes self-repair?
I think self-repair might have lower free energy, in the sense that if you had two configurations of the weights, which “compute the same thing” but one of them has self-repair for a given behaviour and one doesn’t, then the one with self-repair will have lower free energy (which is just a way of saying that if you integrate the Bayesian posterior in a neighbourhood of both, the one with self-repair gives you a higher number, i.e. its preferred).
That intuition is based on some understanding of what controls the asymptotic (in the dataset size) behaviour of the free energy (which is -log(integral of posterior over region)) and the example in that post. But to be clear it’s just intuition. It should be possible to empirically check this somehow but it hasn’t been done.
Basically the argument is self-repair ⇒ robustness of behaviour to small variations in the weights ⇒ low local learning coefficient ⇒ low free energy ⇒ preferred
I think by “specifically” you might be asking for a mechanism which causes the self-repair to develop? I have no idea.