Well, it certainly has mutual information with the training data, even if it only acts as a classifier (actually, classifiers can be seen as inverse generative models, so there is some generative-ish information there, as well). From that perspective, your arguments certainly hold. Although, I’m not sure if “mutual information” is precisely what you’re going for, either. Yes, I agree, I should have tabooed “knowledge” in how I read it.
Well, it certainly has mutual information with the training data, even if it only acts as a classifier (actually, classifiers can be seen as inverse generative models, so there is some generative-ish information there, as well). From that perspective, your arguments certainly hold. Although, I’m not sure if “mutual information” is precisely what you’re going for, either. Yes, I agree, I should have tabooed “knowledge” in how I read it.