This is certainly an interesting line of investigation, and as far as I know it’s still unsolved. There’s still some things that need to be proven to get to an Occam-like distribution, which I mentioned the last time this came up.
Specifically, I haven’t seen any good justification for the assumption about there being a finite number of correct hypotheses (or any variant on this assumption; there’s lots to choose from), since there could be hypothesis-schemas that generate infinite numbers of true hypotheses. A full analysis of this question would probably have to account for those, in a way that assigns all the hypotheses from a particular schema complexity of the schema. I also haven’t seen anyone argue from “prior probabilities decrease monotonically” to “prior probabilities decrease exponentially”; it seems natural since the size of the space of hypotheses of length n increases exponentially with n, but I don’t know how to prove that they don’t instead decrease faster or slower than exponentially, or what other assumptions may be necessary.
You don’t need finite, only countable. Assuming that our conception of the universe is approximately correct, we are only capable of generating a countable set of hypotheses.
Huh? There are only countably many statements in any string-based language, so this includes decidedly non-Occamian scenarios like every statement being true, or every statement being true with the same probability.
I’m confused. What is the countable set of hypotheses you are considering? My claim is merely that if you have hypotheses H1, H2, …, then p(Hi) > 1/n for at most n-1 values of i. This can be thought of as a weak form of Occam’s razor.
In what sense is “every statement being true” a choice of a countable set of hypotheses?
I think maybe the issue is that we are using hypothesis in a different sense. In my case a hypothesis is a complete model of the world, so it is not possible for multiple hypotheses to be true. You can marginalize out / observe a bunch of variables to talk about a subset of the world, but your hypotheses should still be mutually exclusive.
I think the hypotheses are assumed to be mutually exclusive. For example you could have a long list of possible sets of laws of physics, at most one is true of this universe.
Right, that’s another way of stating the same assumption. But we usually apply Occam’s razor to statements in languages that admit sets of non-mutually-exclusive hypotheses of infinite size. So you’d need to somehow collapse or deduplicate those in a way that makes them finite.
I find I mostly apply Occam’s razor to mutually exclusive hypotheses, e.g. explanation A of phenomenon X is better than explanation B because it is simpler.
This is certainly an interesting line of investigation, and as far as I know it’s still unsolved. There’s still some things that need to be proven to get to an Occam-like distribution, which I mentioned the last time this came up.
Specifically, I haven’t seen any good justification for the assumption about there being a finite number of correct hypotheses (or any variant on this assumption; there’s lots to choose from), since there could be hypothesis-schemas that generate infinite numbers of true hypotheses. A full analysis of this question would probably have to account for those, in a way that assigns all the hypotheses from a particular schema complexity of the schema. I also haven’t seen anyone argue from “prior probabilities decrease monotonically” to “prior probabilities decrease exponentially”; it seems natural since the size of the space of hypotheses of length n increases exponentially with n, but I don’t know how to prove that they don’t instead decrease faster or slower than exponentially, or what other assumptions may be necessary.
You don’t need finite, only countable. Assuming that our conception of the universe is approximately correct, we are only capable of generating a countable set of hypotheses.
Huh? There are only countably many statements in any string-based language, so this includes decidedly non-Occamian scenarios like every statement being true, or every statement being true with the same probability.
I’m confused. What is the countable set of hypotheses you are considering? My claim is merely that if you have hypotheses H1, H2, …, then p(Hi) > 1/n for at most n-1 values of i. This can be thought of as a weak form of Occam’s razor.
In what sense is “every statement being true” a choice of a countable set of hypotheses?
I think maybe the issue is that we are using hypothesis in a different sense. In my case a hypothesis is a complete model of the world, so it is not possible for multiple hypotheses to be true. You can marginalize out / observe a bunch of variables to talk about a subset of the world, but your hypotheses should still be mutually exclusive.
I think the hypotheses are assumed to be mutually exclusive. For example you could have a long list of possible sets of laws of physics, at most one is true of this universe.
Right, that’s another way of stating the same assumption. But we usually apply Occam’s razor to statements in languages that admit sets of non-mutually-exclusive hypotheses of infinite size. So you’d need to somehow collapse or deduplicate those in a way that makes them finite.
I find I mostly apply Occam’s razor to mutually exclusive hypotheses, e.g. explanation A of phenomenon X is better than explanation B because it is simpler.