Yes, but there is a problem of what I called “median complexity of the world descriptions”, which is probably answered somewhere but I don’t know where to look.
In other words, Occam razor doesn’t mean that the simplest explanation is true. It means that the simpler explanations are more probable to be true than more complex ones. The difference between the two definitions is the way how the truth is distributed over complexity of the explanations.
In the first case, the distribution is very steep, so the simplest explanation is more probable than all more complex explanation combined. In the second case, the truth(complexity) function declines slowly, so may be first 100 explanations combined have 0.5 probability, - in that case, it is unlikely that the simplest explanation will be true.
Sure, but the fact that the probability distribution is skewed in favor of simpler (i.e. more “beautiful”) explanations by Occam’s Razor is equivalent to saying that there should be such a bias—after all, bias is essentially just a skewing of one’s probability function. Of course this bias shouldn’t be taken to the extreme of assuming that just because one hypothesis is more beautiful than others, it automatically qualifies as the correct explanation. But discrediting such an extreme mindset doesn’t mean that a mild bias in favor of “beauty” is discredited.
Yes, but there is a problem of what I called “median complexity of the world descriptions”, which is probably answered somewhere but I don’t know where to look.
In other words, Occam razor doesn’t mean that the simplest explanation is true. It means that the simpler explanations are more probable to be true than more complex ones. The difference between the two definitions is the way how the truth is distributed over complexity of the explanations.
In the first case, the distribution is very steep, so the simplest explanation is more probable than all more complex explanation combined. In the second case, the truth(complexity) function declines slowly, so may be first 100 explanations combined have 0.5 probability, - in that case, it is unlikely that the simplest explanation will be true.
This article on Solomonoff induction goes over a lot of the related considerations.
Yes, it is a good post, but doesn’t cover the problem of median complexity directly.
Sure, but the fact that the probability distribution is skewed in favor of simpler (i.e. more “beautiful”) explanations by Occam’s Razor is equivalent to saying that there should be such a bias—after all, bias is essentially just a skewing of one’s probability function. Of course this bias shouldn’t be taken to the extreme of assuming that just because one hypothesis is more beautiful than others, it automatically qualifies as the correct explanation. But discrediting such an extreme mindset doesn’t mean that a mild bias in favor of “beauty” is discredited.