Details can be found in the excellent textbook by Li and Vitanyi.
In this context, “hypothesis” means a computer program that predicts your past experience and then goes on to make a specific prediction about the future.
“X or Y” is not such a computer program—it’s a logical abstraction about computer programs.
Now, one might take two programs that have the same output, and then construct another program that is sorta like “X or Y” that runs both X and Y and then reports only one of their outputs by some pseudo-random process. In which case it might be important to you to know about how you can construct Solomonoff induction using only the shortest program that produces each unique prediction.
Can a thing be simple under one definition of simplicity and not simple under another? The contemporary philosopher Karl R. Popper (1902– 1994) has said that Occam’s razor is without sense, since there is no objective criterion for simplicity. Popper states that every such proposed criterion will necessarily be biased and subjective.
There’s no citation. There’s one Popper book in the references section, LScD, but it doesn’t contain the string “occam” (case insensitive search).
I also searched a whole folder of many Popper books and found nothing mentioning Occam (except it’s mentioned by other people, not Popper, in the Schlipp volumes).
If Popper actually said something about Occam’s razor, I’d like to read it. Any idea what’s going on? This seems like a scholarship problem from Li and Vitanyi. They also dismiss Popper’s solution to the problem of induction as unsatisfactory, with no explanation, argument, cite, etc.
Chapter 7 of LScD is about simplicity, but he does not express there the views that Li and Vitanyi attribute to him. Perhaps he said such things elsewhere, but in LScD he presents his view of simplicity as degree of falsifiability. The main difference I see between Popper and Li-Vitanyi is that Popper did not have the background to look for a mathematical formulation of his ideas.
Which section of the 850 page book contains a clear explanation of this? On initial review they seem to talk about hypotheses, for hundreds of pages, without trying to define them or explain what sorts of things do and do not qualify or how Solomonoff hypotheses do and do not match the common sense meaning of a hypothesis.
I’d rather frame this as good news. The good news is that if you want to learn about Solomonoff induction, the entire first half-and-a-bit of the book is a really excellent resource. It’s like if someone directed you to a mountain of pennies. Yes, you aren’t going to be able to take this mountain of pennies home anytime soon, and that might feel awkward, but it’s not like you’d be materially better off if the mountain was smaller.
If you just want the one-sentence answer, it’s as above—“X or Y” is not a Turing machine. If you want to be able to look the whole edifice over on your own, though, it really will take 200+ pages of work (it took me about 3 months of reading on the train) - starting with prefix-free codes and Kolmogorov complexity, and moving on to sequence prediction and basic Solomonoff induction and the proofs of its nice properties. Then you can get more applied stuff like thinking about how to encode what you actually want to ask in terms of Solomonoff induction, minimum message length prediction and other bounds that hold even if you’re not a hypercomputer, and the universal prior and the proofs that it retains the nice properties of basic Solomonoff induction.
Details can be found in the excellent textbook by Li and Vitanyi.
In this context, “hypothesis” means a computer program that predicts your past experience and then goes on to make a specific prediction about the future.
“X or Y” is not such a computer program—it’s a logical abstraction about computer programs.
Now, one might take two programs that have the same output, and then construct another program that is sorta like “X or Y” that runs both X and Y and then reports only one of their outputs by some pseudo-random process. In which case it might be important to you to know about how you can construct Solomonoff induction using only the shortest program that produces each unique prediction.
Li and Vitanyi write:
There’s no citation. There’s one Popper book in the references section, LScD, but it doesn’t contain the string “occam” (case insensitive search).
I also searched a whole folder of many Popper books and found nothing mentioning Occam (except it’s mentioned by other people, not Popper, in the Schlipp volumes).
If Popper actually said something about Occam’s razor, I’d like to read it. Any idea what’s going on? This seems like a scholarship problem from Li and Vitanyi. They also dismiss Popper’s solution to the problem of induction as unsatisfactory, with no explanation, argument, cite, etc.
Chapter 7 of LScD is about simplicity, but he does not express there the views that Li and Vitanyi attribute to him. Perhaps he said such things elsewhere, but in LScD he presents his view of simplicity as degree of falsifiability. The main difference I see between Popper and Li-Vitanyi is that Popper did not have the background to look for a mathematical formulation of his ideas.
Try searching “parsimony” maybe? Another way to express Occam.
Which section of the 850 page book contains a clear explanation of this? On initial review they seem to talk about hypotheses, for hundreds of pages, without trying to define them or explain what sorts of things do and do not qualify or how Solomonoff hypotheses do and do not match the common sense meaning of a hypothesis.
I’d rather frame this as good news. The good news is that if you want to learn about Solomonoff induction, the entire first half-and-a-bit of the book is a really excellent resource. It’s like if someone directed you to a mountain of pennies. Yes, you aren’t going to be able to take this mountain of pennies home anytime soon, and that might feel awkward, but it’s not like you’d be materially better off if the mountain was smaller.
If you just want the one-sentence answer, it’s as above—“X or Y” is not a Turing machine. If you want to be able to look the whole edifice over on your own, though, it really will take 200+ pages of work (it took me about 3 months of reading on the train) - starting with prefix-free codes and Kolmogorov complexity, and moving on to sequence prediction and basic Solomonoff induction and the proofs of its nice properties. Then you can get more applied stuff like thinking about how to encode what you actually want to ask in terms of Solomonoff induction, minimum message length prediction and other bounds that hold even if you’re not a hypercomputer, and the universal prior and the proofs that it retains the nice properties of basic Solomonoff induction.