In classical philosophy there are two principles to explain the generalization phenomenon. One is Occam’s razor and the other is Popper’s falsifiability. It turns out that by using machine learning arguments one can show that both of them are not very good and that one can generalize violating these principles. There are other justifications for inferences.
...
What happens if it goes to value a which is not zero? Then one can prove that there exists in the space X a subspace X0 with probability measure a, such that subset of training vectors that belong to this subspace can be separated in all possible ways. This means that you cannot generalize
Interesting quotes from the interview:
...