Yeah I agree that it would be even more interesting to look at various complexity parameters. The inspiration here of course is physics: isolating a particle/effective particle (like a neutron in a nucleus) or an interaction between a fixed set of particles, by putting it in a regime where other interactions and groupings drop out. The goto for a physicist is temperature: you can isolate a neutron by putting the nucleus in a very high-temperature environment like a collider where the constituent baryons separate. This (as well as the behavior wrt generality) is the main reason I suggested for “natural degradation” from SLT, as this samples from the tempered distribution and is the most direct analog of varying temperature (putting stuff in a collider). But you can vary other hyperparameters as well. Probably an even more interesting thing to do is to simultaneously do two things with “opposite” behaviors, which I think is what you’re suggesting above. For a cartoon notions of the memorization-generalization “scale” is that if you have low complexity coming from low parameter count/depth or low training time (the latter often behaves similarly to low data diversity), you get simpler “more memorization-y” circuits (I’m planning to talk more about this later in a “learning stories” series—but from work on grokking, leap complexity, etc. people expect later solutions to generalize better. So if you combine this with the tempering “natural degradation” above, you might be able to get rid of behaviors both above and below a range of interest.
You’re right that tempering is not a binary on/off switch. Because of the nature of tempering, you do expect exponential decay of “inefficient” circuits as your temperature gets higher than the “characteristic temp.” of the circuit (this is analogous to how localized particles tend to have exponentially less coupling as they get separated), so it’s not completely unreasonable to “fully turn off” a class of behaviors. But something special in physics that probably doesn’t happen in AI is that the temperature scales relevant for different forces have very high separation (many orders of magnitude), so scales separate very clearly. In AI, I agree that as you described, tempering will only “partially” turn off many of the behaviors you want to clean up. It’s plausible that for simple circuits there is enough of a separation of characteristic temperature between the circuit and its interactions with other circuits that something approaching the behavior in physics is possible, but for most phenomena I’d guess that your “things decay more messily” picture is more likely.
Yeah I agree that it would be even more interesting to look at various complexity parameters. The inspiration here of course is physics: isolating a particle/effective particle (like a neutron in a nucleus) or an interaction between a fixed set of particles, by putting it in a regime where other interactions and groupings drop out. The goto for a physicist is temperature: you can isolate a neutron by putting the nucleus in a very high-temperature environment like a collider where the constituent baryons separate. This (as well as the behavior wrt generality) is the main reason I suggested for “natural degradation” from SLT, as this samples from the tempered distribution and is the most direct analog of varying temperature (putting stuff in a collider). But you can vary other hyperparameters as well. Probably an even more interesting thing to do is to simultaneously do two things with “opposite” behaviors, which I think is what you’re suggesting above. For a cartoon notions of the memorization-generalization “scale” is that if you have low complexity coming from low parameter count/depth or low training time (the latter often behaves similarly to low data diversity), you get simpler “more memorization-y” circuits (I’m planning to talk more about this later in a “learning stories” series—but from work on grokking, leap complexity, etc. people expect later solutions to generalize better. So if you combine this with the tempering “natural degradation” above, you might be able to get rid of behaviors both above and below a range of interest.
You’re right that tempering is not a binary on/off switch. Because of the nature of tempering, you do expect exponential decay of “inefficient” circuits as your temperature gets higher than the “characteristic temp.” of the circuit (this is analogous to how localized particles tend to have exponentially less coupling as they get separated), so it’s not completely unreasonable to “fully turn off” a class of behaviors. But something special in physics that probably doesn’t happen in AI is that the temperature scales relevant for different forces have very high separation (many orders of magnitude), so scales separate very clearly. In AI, I agree that as you described, tempering will only “partially” turn off many of the behaviors you want to clean up. It’s plausible that for simple circuits there is enough of a separation of characteristic temperature between the circuit and its interactions with other circuits that something approaching the behavior in physics is possible, but for most phenomena I’d guess that your “things decay more messily” picture is more likely.