To be clear, I have only cursory familiarity with SLT. But my thought is we have something like:
Claim: the mechanism that favours generalising circuits involves the fact that symmetries mean they are overrepresented in the parameter space
Claim: generalising algorithms are underrepresented in the parameter space
Which seem to be in tension. Perhaps the synthesis is that only a few of the generalising algorithms are represented, but those that are are represented many times.
Interesting—what SLT prediction do you think is relevant here?
To be clear, I have only cursory familiarity with SLT. But my thought is we have something like:
Claim: the mechanism that favours generalising circuits involves the fact that symmetries mean they are overrepresented in the parameter space
Claim: generalising algorithms are underrepresented in the parameter space
Which seem to be in tension. Perhaps the synthesis is that only a few of the generalising algorithms are represented, but those that are are represented many times.