Solomonoff induction does not talk about how to make optimal tradeoffs in the programs that serve as the hypothesis.
Imagine you want to describe a part of the world that contains a gun. Solomonoff induction would converge on finding the program that perfectly predicts all the possible observations. So this program would be able to predict what sort of observations would I make after I stuff a banana into the muzzle and fire it. But knowing how the banana was splattered around is not the most useful fact about the gun. It is more useful to know that a gun can be used to kill humans and animals. So if you want to store your world model in only n bits of memory, you need to decide which information to put in. And this matters because some information is much more useful than others. So how can we find the world model that gives you the most power over the world, i.e. letting you reach the greatest number of states? Humans have the ability to judge the usefulness of information. You can ask yourself, what sort of knowledge would be most useful for you to learn? Or, What knowledge would be most bad to forget?
Solomonoff induction does not talk about how to make optimal tradeoffs in the programs that serve as the hypothesis.
Imagine you want to describe a part of the world that contains a gun. Solomonoff induction would converge on finding the program that perfectly predicts all the possible observations. So this program would be able to predict what sort of observations would I make after I stuff a banana into the muzzle and fire it. But knowing how the banana was splattered around is not the most useful fact about the gun. It is more useful to know that a gun can be used to kill humans and animals. So if you want to store your world model in only n bits of memory, you need to decide which information to put in. And this matters because some information is much more useful than others. So how can we find the world model that gives you the most power over the world, i.e. letting you reach the greatest number of states? Humans have the ability to judge the usefulness of information. You can ask yourself, what sort of knowledge would be most useful for you to learn? Or, What knowledge would be most bad to forget?