Reading Wolpert’s Paper (no-free-lunch for optimization) (Scihub is your friend), I got the impression that he might in large part be himself responsible for people misunderstanding the implications of his theorem, because he does not understand them himself:
First, if the practitioner has knowledge of problem characteristics, but does not incorporate them into the optimization algorithm, then P(f) is effectively uniform. (Recall that P(f) can be viewed as a statement concerning the practitioner’s choice of optimization algorithms.) In such a case, the NFL theorems establish that there are no formal assurances that the algorithm chosen will be at all effective.
Wolpert does not actually justify this assumption in his paper, but here are some of the interesting results he is getting from them:
In Section VII we present an introduction to the alternative approach to the formal analysis of optimization in which problems are held fixed and one looks at properties across the space of algorithms. Since these results hold in general, they hold for any and all optimization problems and thus are independent of the types of problems one is more or less likely to encounter in the real world. In particular, these results show that there is no a priori justification for using a search algorithm’s observed behavior to date on a particular cost function to predict its future behavior on that function.
I think in the real world, you can use induction. I am going to stick my neck out, claiming that Wolpert uses induction himself.
For an in-depth critique of the weird assumptions in the paper, see here.
Reading Wolpert’s Paper (no-free-lunch for optimization) (Scihub is your friend), I got the impression that he might in large part be himself responsible for people misunderstanding the implications of his theorem, because he does not understand them himself:
Wolpert does not actually justify this assumption in his paper, but here are some of the interesting results he is getting from them:
I think in the real world, you can use induction. I am going to stick my neck out, claiming that Wolpert uses induction himself.
For an in-depth critique of the weird assumptions in the paper, see here.