They don’t seem like a success of any statistical theory to me
In absolute terms you’re correct. In relative terms, they’re an object that at least frequentist theory can begin to analyze (as you point out, statistical learning theory did, somewhat unsuccessfully).
Whereas Bayesian theory would throw up its hands and say it’s not a prior that gets updated, so it’s not worth considering as a statistical estimator. This seems even wronger.
More recent theory can account for them working, somewhat. But it’s about analyzing their properties as estimators (i.e. frequentism) as opposed to framing them in terms of prior/posterior (though there’s plenty of attempts to the latter going around).
In absolute terms you’re correct. In relative terms, they’re an object that at least frequentist theory can begin to analyze (as you point out, statistical learning theory did, somewhat unsuccessfully).
Whereas Bayesian theory would throw up its hands and say it’s not a prior that gets updated, so it’s not worth considering as a statistical estimator. This seems even wronger.
More recent theory can account for them working, somewhat. But it’s about analyzing their properties as estimators (i.e. frequentism) as opposed to framing them in terms of prior/posterior (though there’s plenty of attempts to the latter going around).
I think this comment of mine serves well as a response to this as well as the comment it was originally responding to.