Here’s my understanding of the dialog, which (as I read it) is not particularly critical of the Solomonoff prior, if that is what cousin_it meant by “casts a shadow”.
(Background knowledge) Shalizi understands “Occam’s Razor” to be something like “In order to reach the truth, among the theories compatible with the evidence, chose the simplest”.
There is a claim that he wishes to refute. The claim is that a certain result is an explanation or proof of Occam’s Razor. The result says that if one finds a simple classification rule which works well in-sample, then it is highly probable that it will continue to work well out-of-sample.
This is a failure of relevance. Occam’s Razor, as Shalizi understands it, is a way of obtaining TRUTH, but the proof only concludes something about GENERALIZATION PERFORMANCE. To illustrate the difference, he points to an example where, in order to increase generalization performance, one might decrease truth.
Shalizi contrasts the algorithmic information theory proof with Kevin T. Kelly’s Ockham Efficiency Theorem, which seems to Shalizi more productive. In particular, Kevin T. Kelly’s formalization does talk about truth rather than generalization performance.
Finally, Shalizi provides an alternative ending to the algorithmic information theory proof. If instead of choosing the simplest classification rule, one chose the simplest rule within a sparse random subset of rules (even a non-computable random subset), then you could still conclude a bound on generalization performance. By providing an alternative ending, he has constructed an alternative proof. Presumably this alternative proof does NOT seem like it is a formalization of Occam’s Razor. Therefore, the interpretation of the original proof as demonstrating some version of Occam’s Razor must also be mistaken.
To sum up, Shalizi is arguing that a certain rhetorical/motivational/interpretational notion which often occurs near a specific proof is wrong. I don’t think he’s concluding anything at all about the Solomonoff prior.
Here’s my understanding of the dialog, which (as I read it) is not particularly critical of the Solomonoff prior, if that is what cousin_it meant by “casts a shadow”.
(Background knowledge) Shalizi understands “Occam’s Razor” to be something like “In order to reach the truth, among the theories compatible with the evidence, chose the simplest”.
There is a claim that he wishes to refute. The claim is that a certain result is an explanation or proof of Occam’s Razor. The result says that if one finds a simple classification rule which works well in-sample, then it is highly probable that it will continue to work well out-of-sample.
This is a failure of relevance. Occam’s Razor, as Shalizi understands it, is a way of obtaining TRUTH, but the proof only concludes something about GENERALIZATION PERFORMANCE. To illustrate the difference, he points to an example where, in order to increase generalization performance, one might decrease truth.
Shalizi contrasts the algorithmic information theory proof with Kevin T. Kelly’s Ockham Efficiency Theorem, which seems to Shalizi more productive. In particular, Kevin T. Kelly’s formalization does talk about truth rather than generalization performance.
Finally, Shalizi provides an alternative ending to the algorithmic information theory proof. If instead of choosing the simplest classification rule, one chose the simplest rule within a sparse random subset of rules (even a non-computable random subset), then you could still conclude a bound on generalization performance. By providing an alternative ending, he has constructed an alternative proof. Presumably this alternative proof does NOT seem like it is a formalization of Occam’s Razor. Therefore, the interpretation of the original proof as demonstrating some version of Occam’s Razor must also be mistaken.
To sum up, Shalizi is arguing that a certain rhetorical/motivational/interpretational notion which often occurs near a specific proof is wrong. I don’t think he’s concluding anything at all about the Solomonoff prior.