Flying by the intuitive seat of my pants, it seems that minimum message length of theory + data is too kind to moderately complex theories that explain extremely complex data, as compared to very simple theories that explain the data almost as well. Maybe the product of the two MML’s, theory and data, would be more apt.
For what it’s worth, some of this was new to me, despite Kolmogorov Complexity being somewhat familiar.
Kolmogorov complexity is somewhat familiar to me too.
Optimizing for the sum of the lengths of the theory and the compressed data seems to be the right thing to do because one can always store part of the theory in the compressed data. This doesn’t change the sum (theory + compressed data). Optimizing for the product might reward this behavior too much.
Flying by the intuitive seat of my pants, it seems that minimum message length of theory + data is too kind to moderately complex theories that explain extremely complex data, as compared to very simple theories that explain the data almost as well. Maybe the product of the two MML’s, theory and data, would be more apt.
For what it’s worth, some of this was new to me, despite Kolmogorov Complexity being somewhat familiar.
Kolmogorov complexity is somewhat familiar to me too.
Optimizing for the sum of the lengths of the theory and the compressed data seems to be the right thing to do because one can always store part of the theory in the compressed data. This doesn’t change the sum (theory + compressed data). Optimizing for the product might reward this behavior too much.